{ "cells": [ { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "# Metodología para clasificación\n", "### Aprendizaje Automático - Instituto de Computación - UdelaR\n" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "## Metodología para clasificación\n", "\n", "- Indepedientemente del método utilizado, existen etapas comunes para poder hacer aprendizaje supervisado (en particular, clasificación).\n", "\n", "- ¿Cuál es nuestra tarea?: dado un conjunto $X$ de instancias independientes con una cierta distribución $D$, cada una de ellas con una clase $y$ asociada, queremos construir una función de clasificación que, dada una instancia nueva, nos devuelva su clase. \n", "\n", "- Algunas preguntas: ¿cómo aprendo la función?, ¿sobre qué instancias?, ¿cómo evalúo la performance de la función?\n", "\n", "\"Drawing\"\n" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "## Fase 1: Preprocesamiento" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "- Vamos a suponer que, para poder entrenar un clasificador, debemos partir de un conjunto $D = \\{(x_i,y_i)\\}$, llamado conjunto de entrenamiento, donde cada instancia $x_i \\in \\mathbb{R}^n$ y $y_i \\in \\mathbb{R}$ (no todos los algoritmos de aprendizaje necesitan este formato de entrada, es solamente para fijar ideas)\n", "\n", "- (Des) afortunadamente, los conjuntos de datos que generalmente disponemos surgen de sensores, datos ingresados por humanos, fuentes diferentes, etc. Por lo tanto, debemos limpiarlos (_data cleaning_).\n", "\n", "- El formato de los datos originales puede ser diverso: elementos de un conjunto (categóricos), fechas, textos, imágenes, etc. Debemos buscar formas para convertirlos a un formato aceptable por el algoritmo (_data transformation_)." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "- Nuestros datos pueden venir de diferentes fuentes, debemos integrarlos (_data integration_)\n", "\n", "- Puede que, para ser más eficientes en los tiempos de aprendizaje, sea necesario agrupar datos, eliminar atributos o reducir el numero de instancias, buscando no perder infromación (_data reduction_)\n", "\n", "![Data Preprocessing](https://miro.medium.com/max/628/1*d1P90NT33rRKlJT7opFO8w.png)\n", "\n", "Fuente de la imgen: [Data Preprocessing](https://medium.com/@silicon.smile1/data-preprocessing-b1552b4060f3) - Umar Farooq " ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "#### Titanic\n", "\n", "Trabajaremos con un ejemplo utilizando pandas (ya que estamos, importamos otras bibliotecas que probablemente utilicemos). \n", "- Titanic Dataset: listado de pasajeros del Titanic, indicando si sobrevivieron o no. Más detalles [aquí](https://www.kaggle.com/c/titanic). " ] }, { "cell_type": "code", "execution_count": 29, "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [], "source": [ "import numpy as np\n", "import pandas as pd\n", "import sklearn\n", "import sklearn.preprocessing\n", "import sklearn.feature_selection\n", "pd.options.mode.chained_assignment = 'warn' # default='warn'\n", "import graphviz\n" ] }, { "cell_type": "code", "execution_count": 30, "metadata": { "slideshow": { "slide_type": "slide" } }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
row.namespclasssurvivednameageembarkedhome.destroomticketboatsex
011st1Allen, Miss Elisabeth Walton29.0000SouthamptonSt Louis, MOB-524160 L2212female
121st0Allison, Miss Helen Loraine2.0000SouthamptonMontreal, PQ / Chesterville, ONC26NaNNaNfemale
231st0Allison, Mr Hudson Joshua Creighton30.0000SouthamptonMontreal, PQ / Chesterville, ONC26NaN(135)male
341st0Allison, Mrs Hudson J.C. (Bessie Waldo Daniels)25.0000SouthamptonMontreal, PQ / Chesterville, ONC26NaNNaNfemale
451st1Allison, Master Hudson Trevor0.9167SouthamptonMontreal, PQ / Chesterville, ONC22NaN11male
....................................
130813093rd0Zakarian, Mr ArtunNaNNaNNaNNaNNaNNaNmale
130913103rd0Zakarian, Mr MapriederNaNNaNNaNNaNNaNNaNmale
131013113rd0Zenn, Mr PhilipNaNNaNNaNNaNNaNNaNmale
131113123rd0Zievens, ReneNaNNaNNaNNaNNaNNaNfemale
131213133rd0Zimmerman, LeoNaNNaNNaNNaNNaNNaNmale
\n", "

1313 rows × 11 columns

\n", "
" ], "text/plain": [ " row.names pclass survived \\\n", "0 1 1st 1 \n", "1 2 1st 0 \n", "2 3 1st 0 \n", "3 4 1st 0 \n", "4 5 1st 1 \n", "... ... ... ... \n", "1308 1309 3rd 0 \n", "1309 1310 3rd 0 \n", "1310 1311 3rd 0 \n", "1311 1312 3rd 0 \n", "1312 1313 3rd 0 \n", "\n", " name age embarked \\\n", "0 Allen, Miss Elisabeth Walton 29.0000 Southampton \n", "1 Allison, Miss Helen Loraine 2.0000 Southampton \n", "2 Allison, Mr Hudson Joshua Creighton 30.0000 Southampton \n", "3 Allison, Mrs Hudson J.C. (Bessie Waldo Daniels) 25.0000 Southampton \n", "4 Allison, Master Hudson Trevor 0.9167 Southampton \n", "... ... ... ... \n", "1308 Zakarian, Mr Artun NaN NaN \n", "1309 Zakarian, Mr Maprieder NaN NaN \n", "1310 Zenn, Mr Philip NaN NaN \n", "1311 Zievens, Rene NaN NaN \n", "1312 Zimmerman, Leo NaN NaN \n", "\n", " home.dest room ticket boat sex \n", "0 St Louis, MO B-5 24160 L221 2 female \n", "1 Montreal, PQ / Chesterville, ON C26 NaN NaN female \n", "2 Montreal, PQ / Chesterville, ON C26 NaN (135) male \n", "3 Montreal, PQ / Chesterville, ON C26 NaN NaN female \n", "4 Montreal, PQ / Chesterville, ON C22 NaN 11 male \n", "... ... ... ... ... ... \n", "1308 NaN NaN NaN NaN male \n", "1309 NaN NaN NaN NaN male \n", "1310 NaN NaN NaN NaN male \n", "1311 NaN NaN NaN NaN female \n", "1312 NaN NaN NaN NaN male \n", "\n", "[1313 rows x 11 columns]" ] }, "execution_count": 30, "metadata": {}, "output_type": "execute_result" } ], "source": [ "titanic=pd.read_csv('https://raw.githubusercontent.com/pln-fing-udelar/curso_aa/master/data/titanic.csv')\n", "titanic" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Conjunto de entrenamiento y Testeo. Accuracy (Spoiler)\n", "\n", "\"Drawing\"\n", "\n", "\n", "- Separación inicial: conjunto de entrenamiento y de evaluación. \n", "\n", "- Cuantas más instancias para entrenar tengamos, probablemente mejor será nuestro modelo, PERO...\n", "\n", "- Cuantas más intancias para evaluar tengamos, menor será la varianza de nuestros resultados.\n", "\n", "- Usualmente se divide como 80%-20%, o 70%-30%\n", "\n", "- Accuracy: proporción de las instancias del conjunto de evaluación que nuestro modelo clasifica correctamente\n", "\n", "Fuente de la imagen: [About Train, Validation and Test Sets in Machine Learning](https://towardsdatascience.com/train-validation-and-test-sets-72cb40cba9e7) - Tarang Shah" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Titanic: división del conjunto de entrenamiento" ] }, { "cell_type": "code", "execution_count": 31, "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [ { "data": { "text/plain": [ "(984, 10)" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/plain": [ "(329, 10)" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Primero separamos las X de las y. \n", "titanic_X = titanic.drop(['survived'], axis=1, inplace=False)\n", "titanic_y = titanic[['survived']]\n", "\n", "# Construimos los corpus de entrenamiento y test\n", "\n", "from sklearn.model_selection import train_test_split\n", "X_train, X_test, y_train, y_test = train_test_split(titanic_X, titanic_y, test_size=0.25, random_state=33)\n", "\n", "display(X_train.shape)\n", "display(X_test.shape)\n" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Valores faltantes\n", "\n", "\n", "- ¿qué hacemos si algún atributo de alguna instancia no tiene asignado un valor?\n", "\n", "- Opción 1: eliminar instancias. Problema: reduce el dataset.\n", "\n", "- Opción 2: asignar un valor especial (UNK, -1, 0, etc). Esto indica que, si faltan datos, quiere decir algo. \n", "\n", "- Opción 3: asignar el valor medio (o la mediana, o la moda) del atributo en el conjunto de entrenamiento\n", "\n", "- Opción 4: asignar según el método de aprendizaje que estemos utilizando (e.g. lo visto en Árboles de Decisión)\n", "\n", "Observación: estos cambios aplican a todo el dataset, pero cualquier cálculo de estadística debe hacerse sobre el conjunto de entrenamiento \n", "\n", "- Algunos detalles en Python: [How to handle missing data with Python](https://machinelearningmastery.com/handle-missing-data-python/) - Jason Brownlee\n" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Titanic: atributos faltantes\n", "\n", "Sustiuimos las edades que nos faltan por el promedio" ] }, { "cell_type": "code", "execution_count": 32, "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
row.namespclassnameageembarkedhome.destroomticketboatsex
108610873rdOlsen, Master ArthurNaNNaNNaNNaNNaNNaNmale
12131stAubert, Mrs Leontine PaulineNaNCherbourgParis, FranceB-3517477 L69 6s9female
103610373rdMoubarek, Master William GeorgeNaNNaNNaNNaNNaNNaNmale
8338343rdGronnestad, Mr Daniel Danielsen32.0SouthamptonForesvik, Norway Portland, NDNaNNaNNaNmale
110811093rdPaulsson, Master Gosta LeonardNaNNaNNaNNaNNaNNaNmale
.................................
122512263rdStankovic, Mr JovanNaNNaNNaNNaNNaNNaNmale
6586593rdBaclini, Miss HeleneNaNCherbourgSyria New York, NYNaNNaNNaNfemale
5785792ndWatt, Miss Bertha12.0SouthamptonAberdeen / Portland, ORNaNNaN9female
3913922ndDibden, Mr William18.0SouthamptonNew Forest, EnglandNaNNaNNaNmale
104410453rdMurphy, Miss MargaretNaNNaNNaNNaNNaNNaNfemale
\n", "

984 rows × 10 columns

\n", "
" ], "text/plain": [ " row.names pclass name age embarked \\\n", "1086 1087 3rd Olsen, Master Arthur NaN NaN \n", "12 13 1st Aubert, Mrs Leontine Pauline NaN Cherbourg \n", "1036 1037 3rd Moubarek, Master William George NaN NaN \n", "833 834 3rd Gronnestad, Mr Daniel Danielsen 32.0 Southampton \n", "1108 1109 3rd Paulsson, Master Gosta Leonard NaN NaN \n", "... ... ... ... ... ... \n", "1225 1226 3rd Stankovic, Mr Jovan NaN NaN \n", "658 659 3rd Baclini, Miss Helene NaN Cherbourg \n", "578 579 2nd Watt, Miss Bertha 12.0 Southampton \n", "391 392 2nd Dibden, Mr William 18.0 Southampton \n", "1044 1045 3rd Murphy, Miss Margaret NaN NaN \n", "\n", " home.dest room ticket boat sex \n", "1086 NaN NaN NaN NaN male \n", "12 Paris, France B-35 17477 L69 6s 9 female \n", "1036 NaN NaN NaN NaN male \n", "833 Foresvik, Norway Portland, ND NaN NaN NaN male \n", "1108 NaN NaN NaN NaN male \n", "... ... ... ... ... ... \n", "1225 NaN NaN NaN NaN male \n", "658 Syria New York, NY NaN NaN NaN female \n", "578 Aberdeen / Portland, OR NaN NaN 9 female \n", "391 New Forest, England NaN NaN NaN male \n", "1044 NaN NaN NaN NaN female \n", "\n", "[984 rows x 10 columns]" ] }, "execution_count": 32, "metadata": {}, "output_type": "execute_result" } ], "source": [ "X_train" ] }, { "cell_type": "code", "execution_count": 33, "metadata": { "slideshow": { "slide_type": "slide" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Cantidad de instancias sin valor: 517\n" ] }, { "data": { "text/plain": [ "'Valor de mean_age:31.02962141327623'" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Contamos cuántos NaN son\n", "print(\"Cantidad de instancias sin valor: {0}\".format(X_train['age'].isnull().sum()))\n", "\n", "# Vemos el promedio de edad de los sobrevivientes, según la clase\n", "mean_age=X_train[\"age\"].mean()\n", "display('Valor de mean_age:'+str(mean_age))\n", "\n", "# Actualizamos con la mean_age de cada grupo tanto entrenamiento como evaluación\n", "X_train.loc[X_train['age'].isnull(),'age']=mean_age\n", "X_test.loc[X_test['age'].isnull(),'age']=mean_age\n", "\n" ] }, { "cell_type": "code", "execution_count": 34, "metadata": { "slideshow": { "slide_type": "slide" } }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
row.namespclassnameageembarkedhome.destroomticketboatsex
108610873rdOlsen, Master Arthur31.029621NaNNaNNaNNaNNaNmale
12131stAubert, Mrs Leontine Pauline31.029621CherbourgParis, FranceB-3517477 L69 6s9female
103610373rdMoubarek, Master William George31.029621NaNNaNNaNNaNNaNmale
8338343rdGronnestad, Mr Daniel Danielsen32.000000SouthamptonForesvik, Norway Portland, NDNaNNaNNaNmale
110811093rdPaulsson, Master Gosta Leonard31.029621NaNNaNNaNNaNNaNmale
.................................
122512263rdStankovic, Mr Jovan31.029621NaNNaNNaNNaNNaNmale
6586593rdBaclini, Miss Helene31.029621CherbourgSyria New York, NYNaNNaNNaNfemale
5785792ndWatt, Miss Bertha12.000000SouthamptonAberdeen / Portland, ORNaNNaN9female
3913922ndDibden, Mr William18.000000SouthamptonNew Forest, EnglandNaNNaNNaNmale
104410453rdMurphy, Miss Margaret31.029621NaNNaNNaNNaNNaNfemale
\n", "

984 rows × 10 columns

\n", "
" ], "text/plain": [ " row.names pclass name age \\\n", "1086 1087 3rd Olsen, Master Arthur 31.029621 \n", "12 13 1st Aubert, Mrs Leontine Pauline 31.029621 \n", "1036 1037 3rd Moubarek, Master William George 31.029621 \n", "833 834 3rd Gronnestad, Mr Daniel Danielsen 32.000000 \n", "1108 1109 3rd Paulsson, Master Gosta Leonard 31.029621 \n", "... ... ... ... ... \n", "1225 1226 3rd Stankovic, Mr Jovan 31.029621 \n", "658 659 3rd Baclini, Miss Helene 31.029621 \n", "578 579 2nd Watt, Miss Bertha 12.000000 \n", "391 392 2nd Dibden, Mr William 18.000000 \n", "1044 1045 3rd Murphy, Miss Margaret 31.029621 \n", "\n", " embarked home.dest room ticket boat \\\n", "1086 NaN NaN NaN NaN NaN \n", "12 Cherbourg Paris, France B-35 17477 L69 6s 9 \n", "1036 NaN NaN NaN NaN NaN \n", "833 Southampton Foresvik, Norway Portland, ND NaN NaN NaN \n", "1108 NaN NaN NaN NaN NaN \n", "... ... ... ... ... ... \n", "1225 NaN NaN NaN NaN NaN \n", "658 Cherbourg Syria New York, NY NaN NaN NaN \n", "578 Southampton Aberdeen / Portland, OR NaN NaN 9 \n", "391 Southampton New Forest, England NaN NaN NaN \n", "1044 NaN NaN NaN NaN NaN \n", "\n", " sex \n", "1086 male \n", "12 female \n", "1036 male \n", "833 male \n", "1108 male \n", "... ... \n", "1225 male \n", "658 female \n", "578 female \n", "391 male \n", "1044 female \n", "\n", "[984 rows x 10 columns]" ] }, "execution_count": 34, "metadata": {}, "output_type": "execute_result" } ], "source": [ "X_train" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Atributos categóricos\n", "\n", "- Los atributos categóricos son atributos cuyos valores pertenecen a un conjunto discreto y finito (y, a veces, no numérico) \n", "\n", "- Opción 1: Cuando tenemos $n$ etiquetas, convertir a valores enteros en el rango $[0.. n-1]$. Problema: sigue siendo discreto, e induce un orden entre las etiquetas. Lo primero puede puede perjudicar a algoritmos que asumen valores continuos, lo segundo puede no representar la realidad.\n", "\n", "- Opción 2: one-hot-encoding. Creamos tantos atributos nuevos como etiquetas diferentes haya. En cada instancia, si el valor del atributo original es $i$, el atributo correspondiente al $i$-ésimo valor valdrá 1, y el resto valdrán 0." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Titanic: codificación de atributos categóricos\n", "\n" ] }, { "cell_type": "code", "execution_count": 35, "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [], "source": [ "# Creamos un labelEncoder utilizando scikit-learn\n", "le=sklearn.preprocessing.LabelEncoder()\n", "# Obtenemos las clases a partir de los valores del conjunto de entrenamiento\n", "le.fit(titanic['sex'])\n", "# Mostramos las clases obtenidas\n", "le.classes_\n", "# Ajustamos el campo sex, transformándolo\n", "\n", "X_train['sex'] = le.transform(X_train['sex'])\n", "X_test['sex'] = le.transform(X_test['sex'])" ] }, { "cell_type": "code", "execution_count": 27, "metadata": { "slideshow": { "slide_type": "slide" } }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
row.namespclassnameageembarkedhome.destroomticketboatsex
108610873rdOlsen, Master Arthur31.029621NaNNaNNaNNaNNaN1
12131stAubert, Mrs Leontine Pauline31.029621CherbourgParis, FranceB-3517477 L69 6s90
103610373rdMoubarek, Master William George31.029621NaNNaNNaNNaNNaN1
8338343rdGronnestad, Mr Daniel Danielsen32.000000SouthamptonForesvik, Norway Portland, NDNaNNaNNaN1
110811093rdPaulsson, Master Gosta Leonard31.029621NaNNaNNaNNaNNaN1
.................................
122512263rdStankovic, Mr Jovan31.029621NaNNaNNaNNaNNaN1
6586593rdBaclini, Miss Helene31.029621CherbourgSyria New York, NYNaNNaNNaN0
5785792ndWatt, Miss Bertha12.000000SouthamptonAberdeen / Portland, ORNaNNaN90
3913922ndDibden, Mr William18.000000SouthamptonNew Forest, EnglandNaNNaNNaN1
104410453rdMurphy, Miss Margaret31.029621NaNNaNNaNNaNNaN0
\n", "

984 rows × 10 columns

\n", "
" ], "text/plain": [ " row.names pclass name age \\\n", "1086 1087 3rd Olsen, Master Arthur 31.029621 \n", "12 13 1st Aubert, Mrs Leontine Pauline 31.029621 \n", "1036 1037 3rd Moubarek, Master William George 31.029621 \n", "833 834 3rd Gronnestad, Mr Daniel Danielsen 32.000000 \n", "1108 1109 3rd Paulsson, Master Gosta Leonard 31.029621 \n", "... ... ... ... ... \n", "1225 1226 3rd Stankovic, Mr Jovan 31.029621 \n", "658 659 3rd Baclini, Miss Helene 31.029621 \n", "578 579 2nd Watt, Miss Bertha 12.000000 \n", "391 392 2nd Dibden, Mr William 18.000000 \n", "1044 1045 3rd Murphy, Miss Margaret 31.029621 \n", "\n", " embarked home.dest room ticket boat sex \n", "1086 NaN NaN NaN NaN NaN 1 \n", "12 Cherbourg Paris, France B-35 17477 L69 6s 9 0 \n", "1036 NaN NaN NaN NaN NaN 1 \n", "833 Southampton Foresvik, Norway Portland, ND NaN NaN NaN 1 \n", "1108 NaN NaN NaN NaN NaN 1 \n", "... ... ... ... ... ... ... \n", "1225 NaN NaN NaN NaN NaN 1 \n", "658 Cherbourg Syria New York, NY NaN NaN NaN 0 \n", "578 Southampton Aberdeen / Portland, OR NaN NaN 9 0 \n", "391 Southampton New Forest, England NaN NaN NaN 1 \n", "1044 NaN NaN NaN NaN NaN 0 \n", "\n", "[984 rows x 10 columns]" ] }, "execution_count": 27, "metadata": {}, "output_type": "execute_result" } ], "source": [ "X_train" ] }, { "cell_type": "code", "execution_count": 36, "metadata": { "slideshow": { "slide_type": "slide" } }, "outputs": [ { "data": { "text/plain": [ "[array(['1st', '2nd', '3rd'], dtype=object)]" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Utilizamos scikit-learn para crear un one-hot-encoder\n", "ohe=sklearn.preprocessing.OneHotEncoder(sparse_output=False)\n", "\n", "# Obtenemos las categorías a partir de los datos de entrenamiento\n", "# Observemos que aquí utilizo todo el dataset, porque la decisión no involucra el aprendizaje\n", "ohe.fit(titanic['pclass'].to_numpy().reshape(-1,1))\n", "display(ohe.categories_)\n", "\n", "# Obtenemos los nuevos valores a partir del valor original\n", "new_train=ohe.transform(X_train['pclass'].to_numpy().reshape(-1,1))\n", "new_test =ohe.transform(X_test['pclass'].to_numpy().reshape(-1,1))\n", "\n", "\n", "# Creamos nuevos atributos\n", "X_train['class_1st']=new_train[:,0]\n", "X_train['class_2nd']=new_train[:,1]\n", "X_train['class_3rd']=new_train[:,2]\n", "X_test['class_1st']=new_test[:,0]\n", "X_test['class_2nd']=new_test[:,1]\n", "X_test['class_3rd']=new_test[:,2]\n" ] }, { "cell_type": "code", "execution_count": 12, "metadata": { "slideshow": { "slide_type": "slide" } }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
row.namespclassnameageembarkedhome.destroomticketboatsexclass_1stclass_2ndclass_3rd
108610873rdOlsen, Master Arthur31.029621NaNNaNNaNNaNNaNmale0.00.01.0
12131stAubert, Mrs Leontine Pauline31.029621CherbourgParis, FranceB-3517477 L69 6s9female1.00.00.0
103610373rdMoubarek, Master William George31.029621NaNNaNNaNNaNNaNmale0.00.01.0
8338343rdGronnestad, Mr Daniel Danielsen32.000000SouthamptonForesvik, Norway Portland, NDNaNNaNNaNmale0.00.01.0
110811093rdPaulsson, Master Gosta Leonard31.029621NaNNaNNaNNaNNaNmale0.00.01.0
..........................................
122512263rdStankovic, Mr Jovan31.029621NaNNaNNaNNaNNaNmale0.00.01.0
6586593rdBaclini, Miss Helene31.029621CherbourgSyria New York, NYNaNNaNNaNfemale0.00.01.0
5785792ndWatt, Miss Bertha12.000000SouthamptonAberdeen / Portland, ORNaNNaN9female0.01.00.0
3913922ndDibden, Mr William18.000000SouthamptonNew Forest, EnglandNaNNaNNaNmale0.01.00.0
104410453rdMurphy, Miss Margaret31.029621NaNNaNNaNNaNNaNfemale0.00.01.0
\n", "

984 rows × 13 columns

\n", "
" ], "text/plain": [ " row.names pclass name age \\\n", "1086 1087 3rd Olsen, Master Arthur 31.029621 \n", "12 13 1st Aubert, Mrs Leontine Pauline 31.029621 \n", "1036 1037 3rd Moubarek, Master William George 31.029621 \n", "833 834 3rd Gronnestad, Mr Daniel Danielsen 32.000000 \n", "1108 1109 3rd Paulsson, Master Gosta Leonard 31.029621 \n", "... ... ... ... ... \n", "1225 1226 3rd Stankovic, Mr Jovan 31.029621 \n", "658 659 3rd Baclini, Miss Helene 31.029621 \n", "578 579 2nd Watt, Miss Bertha 12.000000 \n", "391 392 2nd Dibden, Mr William 18.000000 \n", "1044 1045 3rd Murphy, Miss Margaret 31.029621 \n", "\n", " embarked home.dest room ticket boat \\\n", "1086 NaN NaN NaN NaN NaN \n", "12 Cherbourg Paris, France B-35 17477 L69 6s 9 \n", "1036 NaN NaN NaN NaN NaN \n", "833 Southampton Foresvik, Norway Portland, ND NaN NaN NaN \n", "1108 NaN NaN NaN NaN NaN \n", "... ... ... ... ... ... \n", "1225 NaN NaN NaN NaN NaN \n", "658 Cherbourg Syria New York, NY NaN NaN NaN \n", "578 Southampton Aberdeen / Portland, OR NaN NaN 9 \n", "391 Southampton New Forest, England NaN NaN NaN \n", "1044 NaN NaN NaN NaN NaN \n", "\n", " sex class_1st class_2nd class_3rd \n", "1086 male 0.0 0.0 1.0 \n", "12 female 1.0 0.0 0.0 \n", "1036 male 0.0 0.0 1.0 \n", "833 male 0.0 0.0 1.0 \n", "1108 male 0.0 0.0 1.0 \n", "... ... ... ... ... \n", "1225 male 0.0 0.0 1.0 \n", "658 female 0.0 0.0 1.0 \n", "578 female 0.0 1.0 0.0 \n", "391 male 0.0 1.0 0.0 \n", "1044 female 0.0 0.0 1.0 \n", "\n", "[984 rows x 13 columns]" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "X_train" ] }, { "cell_type": "code", "execution_count": 37, "metadata": { "slideshow": { "slide_type": "slide" } }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
agesexclass_1stclass_2ndclass_3rd
108631.02962110.00.01.0
1231.02962101.00.00.0
103631.02962110.00.01.0
83332.00000010.00.01.0
110831.02962110.00.01.0
..................
122531.02962110.00.01.0
65831.02962100.00.01.0
57812.00000000.01.00.0
39118.00000010.01.00.0
104431.02962100.00.01.0
\n", "

984 rows × 5 columns

\n", "
" ], "text/plain": [ " age sex class_1st class_2nd class_3rd\n", "1086 31.029621 1 0.0 0.0 1.0\n", "12 31.029621 0 1.0 0.0 0.0\n", "1036 31.029621 1 0.0 0.0 1.0\n", "833 32.000000 1 0.0 0.0 1.0\n", "1108 31.029621 1 0.0 0.0 1.0\n", "... ... ... ... ... ...\n", "1225 31.029621 1 0.0 0.0 1.0\n", "658 31.029621 0 0.0 0.0 1.0\n", "578 12.000000 0 0.0 1.0 0.0\n", "391 18.000000 1 0.0 1.0 0.0\n", "1044 31.029621 0 0.0 0.0 1.0\n", "\n", "[984 rows x 5 columns]" ] }, "execution_count": 37, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Elminamos atributos que no vamos a utilizar para el aprendizaje\n", "# Esto podríamos afinarlo usando Feature Selection\n", "X_train.drop(['row.names','pclass', 'name', 'embarked', 'home.dest', 'room', 'ticket', 'boat'], axis=1, inplace=True)\n", "X_test.drop(['row.names','pclass', 'name', 'embarked', 'home.dest', 'room', 'ticket', 'boat'], axis=1, inplace=True)\n", "\n", "X_train" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Ingeniería de atributos - Textos\n", "\n", "- Método tradicional: Bag of Words (BOW): a partir de un vocabulario (lista de palabras del lenguaje), \n", "construimos un vector con un atributo por palabra. Valores que toma:\n", "\n", " - 1/0: 1 indica que la palabra existe en el texto, 0 que no. \n", " - Cantidad de ocurrencias de la palabra en el texto (eventualmente normalizada, dividiendo sobre el total de palabras del documento)\n", " - tf-idf: pondera la frecuencia de la palabra viendo qué tan común es en general (un valor alto indica que la palabra es común en el texto de la instancia, pero rara en el dataset).\n", " \n", " - $ tf = \\frac{count}{total}$, siendo $count$ el número de ocurrencias de la palabra en el texto, y $total$ el número total de palabras en el texto\n", " - $ idf = \\log\\frac{N}{n}$, siendo $N$ el número de instancias del conjunto, y $n$ el número de instancias donde la palabra aparece en el texto\n", " - $tf.idf = td \\times idf$\n" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Ingeniería de atributos - Textos\n", "\n", "Ejercicio: supongamos que la palabra \"the\" aparece en 98 de 100 documentos (instancias), y en mi instancia aparece 9 veces (el documento tiene 200 palabras). Análogamente \"computer\" aparece 3 veces en mi texto, y en 8 de 100 instancias. Construya los vectores según los criterios descritos. El total de palabras distintas en el corpus es 850, pero solamente vamos a utilizar las 300 más comunes (es decir que el vector de cada instancia tendrá 300 elementos). \n", "- Valor para la posición correspondiente a la palabra \"the\":\n", " - Si usamos 1/0: 1 (la palabra aparecen en la instancia)\n", " - Si usamos cant. ocurrencias: 9\n", " - Si usamos cant. ocurrencias, normalizado: 9/200 = 0.045\n", " - Si usamos tf-idf: $\\frac{9}{200} \\cdot \\log \\frac{100}{98} = 9.09 \\times 10^{-4}$\n", "\n", "- Valor para la posición correspondiente a la palabra \"computer\":\n", " - Si usamos 1/0: 1 (la palabra aparecen en la instancia)\n", " - Si usamos cant. ocurrencias: 3\n", " - Si usamos cant. ocurrencias, normalizado: 3/200 = 0.015\n", " - Si usamos tf-idf: $\\frac{3}{200} \\cdot \\log \\frac{100}{8} = 0.037$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Ingeniería de atributos - Textos\n", "\n", "- Antes de crear los vectores, es usual preprocesar el texto: dividir en tokens, eliminar palabras muy comunes (stop words), hacer lematización (buscar representantes comunes a varias palabras relacionadas). \n", "- El enfoque BOW no tiene en cuenta el orden de las palabras. Una mejora: en vez de palabras, contar n-gramas (secuencias de n palabras). A este enfoque se lo llama también bag-of-ngrams.\n", "- Podemos imaginar a una representación de conteo normalizado BOW como una suma de vectores one-hot-encoded, uno por cada palabra, donde en la posición de la palabra está el valor correspondiente al conteo, y el resto de las posiciones son 0. \n", "\n", "$$\n", "x = \\frac{1}{|D|}\\sum_{i=1}^{|D|} x^{D_{[i]}}\n", "$$\n", "\n" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Ingeniería de atributos - Textos\n", "\n", "- Problemas de las representaciones anteriores: son muy dispersas (en cada instancia hay muchos atributos con valor 0). No hay noción de similaridad entre palabras, si nuestra base de representación para una palabra es un one-hot-encoding.\n", "- Hoy prácticamente todos los métodos en el estado del arte utilizan _word embeddings_: se generan vectores densors de baja dimensionalidad (50-200 atributos), y con algunas propiedades interesantes, a partir del contexto en el que aparece cada palabra usualmente. \n", "\n", "\n", "Si quieren saber más sobre textos y cómo procesarlos, pueden hacer el curso [Introducción al Procesamiento del Lenguaje Natural](https://eva.fing.edu.uy/course/view.php?id=211), o [Redes Neuronales para Lenguaje Natural](https://eva.fing.edu.uy/course/view.php?id=1758) dictado en esta misma institución, por este mismo grupo de investigación." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Estandarización de atributos\n", "\n", "- Muchos algoritmos en aprendizaje automático (e.g. knn, redes neuronales, PCA) se benefician de que los atributos continuos tengan aproximadamente el mismo orden de magnitud. Esto se debe, por ejemplo, a que se utiliza la distancia euclidiana y se busca que todos los atributos \"pesen\" igual al calcularla. En el caso de algoritmos que utilizan descenso por gradiente, puede haber mucha diferencia en performance. \n", "\n", "- Min-max scaling: este escalado deja los valores en el rango $[0-1]$. Dado un valor $x$, obtenemos:\n", "\n", " $ x_s = \\frac{x - x_{min}}{x_{max} - x_{min}}$\n", " \n", " siendo $x_{min}$ y $x_{max}$ los valores mínimo y máximo respectivamente en el dataset\n", " \n", "- Normalización: se escalan los atributos para que tengan las propiedades de una distribución normal estándar, con $\\mu = 0$ y $\\sigma = 1$. \n", "\n", " $ x_{norm} = \\frac{x_i - \\mu_{i}}{s_i} $\n", " \n", " siendo $\\mu_{i}$ la media y $s_i$ la desviación estándar de la muestra.\n", " \n", " \n", " " ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "Ejercicio: normalice los siguientes valores utilizando los dos métodos anteriores: $\\{85,35,42,8,15, 22\\}$. ¿En qué rango quedan los valores obtenidos por normalización?\n", "\n", "- Valor mínimo: 8\n", "- Valor máximo: 85\n", "- Valores escalados min-max: {1,0.35, 0.44, 0, 0.09, 0.18}\n", "\n", "\n", "\n", "- Media: $\\frac{85+35+42+8+15+22}{6} = 34.50 $\n", "- Desviación estándar: $\\sqrt{(85 -34.5)^2 + \\ldots + (22 - 34.5)^2} = 25.32$\n", "- Valores normalizados: $\\{ 1.99, 0.02, 0.3, -1.05, -0.77, -0.49\\}$\n", "\n", "(Verifique que los nuevos valores tienen media 0 y desviación estándar 1)" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "- Lectura: [About standarization](https://sebastianraschka.com/Articles/2014_about_feature_scaling.html) - Sebastian Rashcka\n", "- Nota: la estandarización, así como la selección de los atributos, debe hacerse durante el entrenamiento (no antes). Véase [este link](https://stats.stackexchange.com/questions/77350/perform-feature-normalization-before-or-within-model-validation) por más detalles.\n", "\n" ] }, { "cell_type": "code", "execution_count": 38, "metadata": { "slideshow": { "slide_type": "slide" } }, "outputs": [ { "data": { "text/plain": [ "[]" ] }, "execution_count": 38, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiMAAAGdCAYAAADAAnMpAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABbIUlEQVR4nO3deVyU5d4/8M/swzIDggw7iOKOO6aguJa5ZJpl2artJm759JxOp+f8Tp2nok6n87glhRVlZtqm6TG3jgKuqaSpuASCgrIMoDDDNsPM3L8/gBFEDRTmhuHzfr3mNczNLF/vlvl4Xd/ruiWCIAggIiIiEolU7AKIiIioY2MYISIiIlExjBAREZGoGEaIiIhIVAwjREREJCqGESIiIhIVwwgRERGJimGEiIiIRCUXu4CmsNlsyM3NhUajgUQiEbscIiIiagJBEGA0GhEQEACp9ObjH+0ijOTm5iI4OFjsMoiIiOg25OTkICgo6Ka/bxdhRKPRAKj5w2i1WpGrISIioqYwGAwIDg62f4/fTLsII3VTM1qtlmGEiIionfmjFosO3cB6oagcb25JQ6XZKnYpREREHVaHDSOCIGDxhuNI3H8BU1fuQ1puqdglERERdUgdNoxIJBL814Qe0GlUyNCXYfqH+7E6JRM2myB2aURERB1Khw0jABDT3QfbF4/CPX18UW0V8PZPZ/DUZ4dRYKgSuzQiIqIOo0OHEQDwclMi4ckhiJvRDy4KGfZlFOHepSnYfipf7NKIiIg6hA4fRoCaKZtH7wrBvxeORL9AD5RUVGPu2lS89sMJVJgtYpdHRETk1BhG6unm447vX4rG3NHdIJEAXx/OwX3L9+HEpRKxSyMiInJaDCPXUcql+POkXvjquWHw06qRWVSOGasOID7pPKxsbiUiImpxDCM3Ed2tM7YvjsHkfn6w2AS8t/0sHv/kEHJLKsUujYiIyKkwjNyCp6sSHz42GO8/1B+uShkOZV7BxKUp2HoiT+zSiIiInMYdh5EuXbpAIpE0usXGxgIA5syZ0+h3w4cPv+PCHUUikWBmZDB+WhiDAcGeMFRZELvuV7zy7W8oM7G5lYiI6E7dcRg5cuQI8vLy7Lddu3YBAGbOnGl/zsSJExs856effrrTj3W4Lp3d8N3cKCwYFw6pBPgu9RImL9uLY9lXxS6NiIioXbvjC+X5+Pg0ePzuu++iW7duGD16tP2YSqWCn5/fnX6U6BQyKf5rQk/EdPfByxuOI/tKBR766CAWj++OeWPDIZPe+kJARERE1FiL9oyYzWasXbsWzzzzTIMr9CUlJUGn06FHjx54/vnnodfrb/k+JpMJBoOhwa0tuSvMCz8tisH9AwJgtQn4YNfvmJVwEDlXKsQujYiIqN1p0TCyadMmlJSUYM6cOfZjkyZNwldffYXdu3fjgw8+wJEjRzBu3DiYTKabvk9cXBw8PDzst+Dg4JYss0V4uCiw/NFB+L9HBsBdJceRC1cxedle/Hj8stilERERtSsSQRBabPOMe++9F0qlElu2bLnpc/Ly8hAaGor169djxowZN3yOyWRqEFYMBgOCg4NRWloKrVbbUuW2mJwrFVi84ThSL9b0j0wfGIC/T4+AVq0QuTIiIiLxGAwGeHh4/OH3d4uNjFy8eBE///wznnvuuVs+z9/fH6GhoUhPT7/pc1QqFbRabYNbWxbs5YoNLwzHy3f3gEwqwabjuZi8bC+OXrgidmlERERtXouFkcTEROh0OkyZMuWWzysuLkZOTg78/f1b6qPbBLlMikV3d8c3L0Yh2MsFl65W4uGPD+Jfu36HxWoTuzwiIqI2q0XCiM1mQ2JiImbPng25/NoCnbKyMrzyyis4ePAgLly4gKSkJEydOhWdO3fGAw880BIf3eYMCe2EnxbGYMbgQNgEYPl/0jHz44O4WFwudmlERERtUouEkZ9//hnZ2dl45plnGhyXyWQ4efIkpk2bhh49emD27Nno0aMHDh48CI1G0xIf3SZp1Ar86+GBWP7oIGjUchzLLsHkZXvxXeoltGCLDhERkVNo0QbW1tLUBpi26HJJJV7ecByHs2r6R+7r74+3p/eDhyubW4mIyLk5vIGVbizQ0wVfPz8c/31vT8ilEvz7RB4mLUvBocxisUsjIiJqExhGHEAmlSB2bDi+eykaXbxdkVtahUdXH8I/tp+F2cLmViIi6tgYRhxoYLAnti6MwSORwRAEYFXSeTz00QFkFpaJXRoREZFoGEYczE0lx3sP9Uf844Ph4aLAiUulmLJ8H9YfzmZzKxERdUgMIyKZ1M8f2xfHILqbNyqrrfjzDyfx0tpfcbXcLHZpREREDsUwIiJ/DxesfXYYXpvUCwqZBNvT8jFxWQr2ZxSJXRoREZHDMIyITCqV4MXR3bBx3gh09XFDgcGExz/5Be/8dAYmi1Xs8oiIiFodw0gbERHoga0LYvD4sBAAQEJKJh748AAy9EaRKyMiImpdDCNtiItShrcf6IeEJ4egk6sCp/MMuG/FPqw9dJHNrURE5LQYRtqgCX39sGPxKMR074yqahv+Z9MpPL/mKIrLTGKXRkRE1OIYRtoonVaNL56+C3+9rw+UMil+PqPHvUv3IumcXuzSiIiIWhTDSBsmlUrw7Mgw/Dh/BHr4uqOozIQ5iUfw5pY0VFWzuZWIiJwDw0g70Ntfi83zR2J2VCgAIHH/BUz/cD/O5bO5lYiI2j+GkXZCrZDhzWkRSJwzFJ3dlTibb8TUlfvw+f4sNrcSEVG7xjDSzoztpcO2RaMwtqcPzBYb3thyGnMSj0BvrBK7NCIiotvCMNIO+WhU+GzOUPx9Wl+o5FIk/16ISUv34j9nCsQujYiIqNkYRtopiUSCp6K6YMuCkejlp0FxuRnPfnEUf910CpVmNrcSEVH7wTDSzvXw1WBT7Ag8OzIMAPDloYuYunIf0nJLRa6MiIioaRhGnIBaIcNf7+uDNc/cBR+NChn6Mjzw4QF8sjcTNhubW4mIqG1jGHEio3r4YMfiUbinjy/MVhve2noGT312GAUGNrcSEVHbxTDiZLzclEh4cgjeeaAf1Aop9mUU4d6lKdiRli92aURERDd0R2HkjTfegEQiaXDz8/Oz/14QBLzxxhsICAiAi4sLxowZg7S0tDsumm5NIpHgsWEh+PeCGPQN0KKkohovfpmK1344iQqzRezyiIiIGrjjkZG+ffsiLy/Pfjt58qT9d//4xz/wr3/9CytXrsSRI0fg5+eHe+65B0Yjdw51hHCdOzbOG4EXR3eFRAJ8fTgb9y3fh5OX2NxKRERtxx2HEblcDj8/P/vNx8cHQM2oyNKlS/H6669jxowZiIiIwBdffIGKigqsW7fujgunplHKpXhtUm989eww+GnVyCwqxwOr9iM+6TysbG4lIqI24I7DSHp6OgICAhAWFoZZs2YhMzMTAJCVlYX8/HxMmDDB/lyVSoXRo0fjwIEDd/qx1EzR4Z2xfXEMJkX4wWIT8N72s3j8k0PILakUuzQiIurg7iiMDBs2DGvWrMGOHTuwevVq5OfnIzo6GsXFxcjPr2mY9PX1bfAaX19f++9uxmQywWAwNLjRnfN0VWLV44Pxjwf7w1Upw6HMK5i0bC+2nsgTuzQiIurA7iiMTJo0CQ8++CD69euHu+++G1u3bgUAfPHFF/bnSCSSBq8RBKHRsevFxcXBw8PDfgsODr6TMqkeiUSCh4cGY+vCGAwI8kBpZTVi1/2K//72N5SZ2NxKRESO16JLe93c3NCvXz+kp6fbV9VcPwqi1+sbjZZc77XXXkNpaan9lpOT05JlEoCwzm747qVozB8bDokE+Db1EqYs34tj2VfFLo2IiDqYFg0jJpMJZ86cgb+/P8LCwuDn54ddu3bZf282m5GcnIzo6Ohbvo9KpYJWq21wo5ankEnxyr09sf754QjwUONicQUe+uggVvwnnc2tRETkMHcURl555RUkJycjKysLv/zyCx566CEYDAbMnj0bEokEixcvxjvvvIONGzfi1KlTmDNnDlxdXfHYY4+1VP3UAoZ19ca2xaNwX39/WG0CPtj1O2YlHETOlQqxSyMiog5AficvvnTpEh599FEUFRXBx8cHw4cPx6FDhxAaGgoA+NOf/oTKykrMmzcPV69exbBhw7Bz505oNJoWKZ5ajoeLAiseHYRxvXT4fz+m4ciFq5i8bC/eeiAC0wYGil0eERE5MYkgCG1+PN5gMMDDwwOlpaWcsnGA7OIKLN5wDL9mlwAAHhgUiDen9YVWrRC3MCIialea+v3Na9NQIyHervjmxSgsGt8dUgmw8dhlTF62F0cvXBG7NCIickIMI3RDcpkUL9/TA9/OjUJQJxdculqJhz8+iH/t+h0Wq03s8oiIyIkwjNAtDQn1wrZFMZgxKBA2AVj+n3TM/PggsovZ3NqWmC02XC6pxIlLJdwvhojaHfaMUJP9ePwy/mfTKRirLHBTyvD3aRGYMTjwDzexo9tnslhRaDShwGBCobEKBQYT9PZ7E/SGKuiNJlwpN9tfo1HLMSe6C54eEQYvN6WI1RNRR9fU72+GEWqWS1crsGTDbzhc2z9yX39/vD29Hzxc2dzaHFXVNSFDb6yC3mBCQW2oqAsb+tr7qxXVTX5PhUwCN5UcJbWvcVXK8MTwUDwXEwadRt1afxQioptiGKFWY7UJiE/KwP/9XLM5WoCHGv96ZCCGd/UWuzTR1YWMa+Hi2n394yXNCBlKmRQ6rQo6jQq+WjV0GhV0tfe+WjV0WhV8NWp4uiogCMDO0/lYsTsDabk113RSyqV4dGgwXhjdDYGeLq31RyciaoRhhFrd8ZwSLFp/DBeLKyCRAPPGdMPiu3tAIXO+VqSqamvNCMZ1Ixn660JHaWUzQoZc2iBg+GrV8Km999WqoNPU3Hu4KJo9FSYIApLOFWL57nQcq12irZBJ8ODgILw0phtCvd2a9X5ERLeDYYQcotxkwZtb0vDN0UsAgP5BHlg2axDCOrePL7tKs7VeD0ZVo2mSAkNN4DBUNb0pVCWX2kcrdLWhov7juvBxOyGjuQRBwMHzxVixOwMHM4sBAFIJcP+AAMSODUd3X25ASESth2GEHOqnk3l47YeTKK2shotChjfu74OHI4NFa26tMFuu68WouuH0ibEZIUOtkNpHK3T1gobvdfdaF3mbbOo9euEKVu7JQNK5QgCARAJM7OuH2LHhiAj0ELk6InJGDCPkcHmllViy4Tf738An9vVD3Ix+6NSCKzrKTZYGYaLBNEntNEqhwQRjM5a3qhXSmqkRjRo+DUYwroUMH40aWnXbDBnNdfJSKT7ck4HtadeuqD2ulw6xY8MxJLSTiJURkbNhGCFRWG0CVu/NxAc7z6HaKsBXq8K/Hh6IEeGdb/m6MpMFesNNpknsj03N2kPDRSGrCRT1mz2vbwLVqqBROUfIaK5z+UasSsrAlt9yUXeR5hHh3pg/tjuGd/XqkOeEiFoWwwiJ6uSlUixafwyZReUAgOdGhqFfkEe9EYxroxp6QxXKzdYmv7erUtag2bMmYDSeOnHvoCGjubKKyhGflIEffr0MS20qGRLaCfPHhWNMDx+eQyK6bQwjJLoKswVvbT2Ddb9kN+n5bjcMGdc1gWrVcFfd0cWm6SYuXa3Ax8mZ2HA0B2ZLzZb/EYFazB/bHRP6+EIqZSghouZhGKE2Y0daPj7ZmwmZVHLDZax1UyYMGW2D3lCF1XszsfZQNiqra0asevi6I3ZsOKb084fcCZduE1HrYBghojtypdyMz/Zl4YsDF+wNwV28XTFvTDimDwqEUs5QQkS3xjBCRC2itLIaaw5cwGf7s+zb0wd6umDu6K6YGRkMtUImcoVE1FYxjBBRiyo3WbDul2x8nJKJojITAMBHo8KLo7risWEhcFVymo2IGmIYIaJWUVVtxTdHc/BR0nnkllYBADq5KvDsyDA8Fd0FWjUvmkhENRhGiKhVmS02bDx2CauSzuNicQUAQKOWY050Fzw9IgxeLbjZHRG1TwwjROQQFqsN/z6Rhw/3ZCBdXwagZi+YJ4aH4rmYMOg0apErJCKxMIwQkUPZbAJ2ns7Hit0ZSMs1AKi5MvGjQ4PxwuhuCPR0EblCInK0pn5/39HavLi4OAwdOhQajQY6nQ7Tp0/HuXPnGjxnzpw5kEgkDW7Dhw+/k48lojZIKpVgYoQ//r1gJBLnDMWgEE+YLTZ8cfAixry/B3/+/gQuFpeLXSYRtUF3NDIyceJEzJo1C0OHDoXFYsHrr7+OkydP4vTp03Bzq7mE/Jw5c1BQUIDExET765RKJby8vJr8ORwZIWp/BEHAwfPFWLE7w37xRKkEuH9AAGLHhqO7r0bkComotYkyTVNYWAidTofk5GSMGjUKQE0YKSkpwaZNm277fRlGiNq3oxeuYOWeDCSdKwQASCQ1V3WOHRuOiEAPkasjotbikGma65WWlgJAo1GPpKQk6HQ69OjRA88//zz0en1LfiwRtXGRXbzw+dN3Ycv8kZjY1w+CAGw7lY/7VuzDM58fQerFq2KXSEQiarGREUEQMG3aNFy9ehV79+61H9+wYQPc3d0RGhqKrKws/PWvf4XFYkFqaipUKtUN38tkMsFkMtkfGwwGBAcHc2SEyEmcyzdiVVIGtvyWi9oLBSO6mzfmjwtHVFdvXimYyEk4fJomNjYWW7duxb59+xAUFHTT5+Xl5SE0NBTr16/HjBkzbvicN954A2+++Waj4wwjRM4lq6gc8UkZ+OHXy7DUppIhoZ0wf1w4xvTwYSghauccGkYWLFiATZs2ISUlBWFhYX/4/O7du+O5557Dq6++esPfc2SEqGO5dLUCHydnYsPRHJgtNgBARKAW88d2x4Q+vpBKGUqI2qOmhpE7upiEIAhYsGABNm7ciKSkpCYFkeLiYuTk5MDf3/+mz1GpVDedwiEi5xPUyRX/Oz0CC8aFY/XeTKw9lI1Tlw2YuzYVPXzdETs2HFP6+UMu45WCie6E1SaguMwEvdGEAkNVg/unokLRy0+cv/Df0cjIvHnzsG7dOvz444/o2bOn/biHhwdcXFxQVlaGN954Aw8++CD8/f1x4cIF/OUvf0F2djbOnDkDjaZpS/u4moaoY7lSbsZn+7LwxYELMJosAIAu3q6YNyYc0wcFQilnKCGqz2K1objcDL2hccjQ13tcVGay92ldb9msgZg2MLBF63LINM3N5nMTExMxZ84cVFZWYvr06Th27BhKSkrg7++PsWPH4n//938RHBzc5M9hGCHqmEorq/HlwQv4dF8WrlZUAwACPV0wd3RXzIwMhlohE7lCotZlsdpQVGaG3liFAoPJfl943ePiW4SM60klQGd3FXy1aug0Kuhq7+/t64c+AS37Hcvt4InIaZSbLFj3SzY+TslEUVlNP5mPRoUXR3XFY8NC4Kq8oxlnIoerttpQVGZqMJKhv25Eo8BgQnG5CU39lpZJJejsrmwUMnyvu/d2V0HmoD4shhEicjpV1VZ8czQHHyWdR25pFQCgk6sCz44Mw1PRXaBVK0SukDq6aqsNhUZT42kSgwkFxpp7vbEKxeXmZoUMH3cVfLUq+GjU8NWqoKu7r/1Zp1XB281xIaOpGEaIyGmZLTZsPHYJq5LO42JxBQBAo5ZjTnQXPD0iDF5uSpErJGdjtthQWFYTLG40TVIXOq5UND1kyKUS+GhUjUYxfOsFDJ1GDS83ZZsLGU3FMEJETs9iteHfJ/Lw4Z4MpOvLAACuShmeGB6K52LCoNOoRa6Q2jqTxWofyWgwTWIwoaDesSvl5ia/p1wqgU6jgo9WDV9NzeiFb124qBc6vFyVTr9snWGEiDoMm03AztP5WLE7A2m5BgCAUi7Fo0OD8cLobgj0dBG5QnI0k8VaOyVyo16MKhTW3tc1RjeFQiaBTqOGj0bVcKqk3iiGr1aFTh0gZDQVwwgRdTiCICDpXCGW707HsewSADVfIA8ODsJLY7oh1NtN3ALpjlVV141k1E6PGKpqRzBqjtX1ZpTcRsioP4Lhq60LHddGMjxdFAwZzcQwQkQdliAIOHi+GCt2Z+BgZjGAmuWM9w8IQOzYcHT3bdoeR+Q4VdVWe6AouO6+/vHSyqaHDKVMWjti0XgZq6/2WvjwdFXw0gOthGGEiAjA0QtXsHJPBpLOFQIAJBJgYl8/xI4NR0Sgh8jVOb9Ks7UmUDToxahCYb3VJQWGKhiqLE1+T6Vc2mjJqs91zZ++WhU8XBgyxMYwQkRUz8lLpfhwTwa2p+Xbj43t6YP547pjSGgnEStrnyrMFntPxs33yaiCsRkhQyWXNmz2vMHUiU7DkNGeMIwQEd3AuXwjViVlYMtvufYdK6O7eWP+uHBEdfXu8F9yFWbLdb0Y14JG/WmTum36m0KtkN602bP+vdZF3uHPv7NhGCEiuoWsonLEJ2Xgh18vw1KbSoaEdsL8ceEY08PH6b4Uy02WG64maTiyYUJZM0OGr1YNX40aPg1GMK6FDB+NGlo1Q0ZHxTBCRNQEl65WICElE+uP5MBssQEAIgK1mD+2Oyb08W3zqyfK6kJG/dUkNwgd5WZrk9/TRSGr3d3zxtuJ62qbPzUqhgy6NYYRIqJm0BuqsHpvJtYeykZldc0Xdw9fd8SODceUfv6Qyxx3pWBBEGpDRuPVJNeHjIpmhAxXpaxBoLBvyHXdMlZ3hgxqIQwjRES34Uq5GZ/ty8IXBy7Y+yK6eLti3phwTB8UCKX89kOJIAgwmiyNrlVyo9BRF4iawl0lrw0YN9+IS6dVw13FCwqSYzGMEBHdgdLKanx58AI+3Zdl36Uz0NMFc0d3xczIYKgVMvtzBUGAocpyw9Uk168yqaq2NbkGjUp+XS/Gja/G6saQQW0UwwgRUQsoN1mw7pdsfJySiaIyEwDAR6NCZGinBldnNVmaETLU8pvuk1F/Qy5XJUMGtW8MI0RELaiq2opvjubgo6TzyC2tuuFzNGr5jZs9GzxmyKCOg2GEiKgVmC02bDuVh+Iyc8OpE40aLkrZH78BUQfS1O9vxnMiomZQyqWYNjBQ7DKInIrj1qoRERER3UC7GBmpm0kyGAwiV0JERERNVfe9/UcdIe0ijBiNRgBAcHCwyJUQERFRcxmNRnh43Pwq2e2igdVmsyE3NxcajaZFdwU0GAwIDg5GTk4OG2NbGc+1Y/A8OwbPs2PwPDtGa55nQRBgNBoREBAAqfTmnSHtYmREKpUiKCio1d5fq9XyX3QH4bl2DJ5nx+B5dgyeZ8dorfN8qxGROmxgJSIiIlExjBAREZGoOnQYUalU+Nvf/gaVSiV2KU6P59oxeJ4dg+fZMXieHaMtnOd20cBKREREzqtDj4wQERGR+BhGiIiISFQMI0RERCQqhhEiIiISVYcNIykpKZg6dSoCAgIgkUiwadMmsUtyOnFxcRg6dCg0Gg10Oh2mT5+Oc+fOiV2W04mPj0f//v3tGxZFRUVh27ZtYpfl9OLi4iCRSLB48WKxS3E6b7zxBiQSSYObn5+f2GU5pcuXL+OJJ56At7c3XF1dMXDgQKSmpjq8jg4bRsrLyzFgwACsXLlS7FKcVnJyMmJjY3Ho0CHs2rULFosFEyZMQHl5udilOZWgoCC8++67OHr0KI4ePYpx48Zh2rRpSEtLE7s0p3XkyBEkJCSgf//+YpfitPr27Yu8vDz77eTJk2KX5HSuXr2KESNGQKFQYNu2bTh9+jQ++OADeHp6OryWdrEdfGuYNGkSJk2aJHYZTm379u0NHicmJkKn0yE1NRWjRo0SqSrnM3Xq1AaP3377bcTHx+PQoUPo27evSFU5r7KyMjz++ONYvXo13nrrLbHLcVpyuZyjIa3svffeQ3BwMBITE+3HunTpIkotHXZkhByvtLQUAODl5SVyJc7LarVi/fr1KC8vR1RUlNjlOKXY2FhMmTIFd999t9ilOLX09HQEBAQgLCwMs2bNQmZmptglOZ3NmzcjMjISM2fOhE6nw6BBg7B69WpRamEYIYcQBAFLlizByJEjERERIXY5TufkyZNwd3eHSqXC3LlzsXHjRvTp00fsspzO+vXr8euvvyIuLk7sUpzasGHDsGbNGuzYsQOrV69Gfn4+oqOjUVxcLHZpTiUzMxPx8fHo3r07duzYgblz52LhwoVYs2aNw2vpsNM05Fjz58/HiRMnsG/fPrFLcUo9e/bE8ePHUVJSgu+//x6zZ89GcnIyA0kLysnJwaJFi7Bz506o1Wqxy3Fq9afQ+/Xrh6ioKHTr1g1ffPEFlixZImJlzsVmsyEyMhLvvPMOAGDQoEFIS0tDfHw8nnrqKYfWwpERanULFizA5s2bsWfPHgQFBYldjlNSKpUIDw9HZGQk4uLiMGDAACxbtkzsspxKamoq9Ho9hgwZArlcDrlcjuTkZCxfvhxyuRxWq1XsEp2Wm5sb+vXrh/T0dLFLcSr+/v6N/sLSu3dvZGdnO7wWjoxQqxEEAQsWLMDGjRuRlJSEsLAwsUvqMARBgMlkErsMpzJ+/PhGKzqefvpp9OrVC6+++ipkMplIlTk/k8mEM2fOICYmRuxSnMqIESMabbfw+++/IzQ01OG1dNgwUlZWhoyMDPvjrKwsHD9+HF5eXggJCRGxMucRGxuLdevW4ccff4RGo0F+fj4AwMPDAy4uLiJX5zz+8pe/YNKkSQgODobRaMT69euRlJTUaDUT3RmNRtOo38nNzQ3e3t7sg2phr7zyCqZOnYqQkBDo9Xq89dZbMBgMmD17ttilOZWXX34Z0dHReOedd/Dwww/j8OHDSEhIQEJCguOLETqoPXv2CAAa3WbPni12aU7jRucXgJCYmCh2aU7lmWeeEUJDQwWlUin4+PgI48ePF3bu3Cl2WR3C6NGjhUWLFoldhtN55JFHBH9/f0GhUAgBAQHCjBkzhLS0NLHLckpbtmwRIiIiBJVKJfTq1UtISEgQpQ6JIAiC4yNQ89hsNuTm5kKj0UAikYhdDhERETWBIAgwGo0ICAiAVHrzNtV2MU2Tm5uL4OBgscsgIiKi25CTk3PLBQzNDiMpKSl4//33kZqairy8PGzcuBHTp0+/5WuSk5OxZMkSpKWlISAgAH/6058wd+7cJn+mRqMBUPOH0Wq1zS2ZiIiIRGAwGBAcHGz/Hr+ZZoeRumu6PP3003jwwQf/8PlZWVmYPHkynn/+eaxduxb79+/HvHnz4OPj06TXA7BPzdRdCIyIiIjajz9qsWh2GGnuNV0++ugjhISEYOnSpQBq1jAfPXoU//znP5scRlqLxWpD9pUKdPVxF7UOIiKijqzVNz07ePAgJkyY0ODYvffei6NHj6K6uvqGrzGZTDAYDA1urSE+6TwmLtuLz/dnoR308RIRETmlVg8j+fn58PX1bXDM19cXFosFRUVFN3xNXFwcPDw87LfWaF4VBAEnLpfCbLHhjS2n8fTnR1Bo5CZRREREjuaQ7eCvnyuqG4W42RzSa6+9htLSUvstJyenVWpKeHII/j6tL1RyKZLOFWLi0hTsPlvQ4p9FREREN9fqYcTPz8++82YdvV4PuVwOb2/vG75GpVLZm1Vbs2lVIpHgqagu2LJgJHr5aVBcbsYznx/F//vxFKqqeZ0JIiIiR2j1MBIVFYVdu3Y1OLZz505ERkZCoVC09sc3SQ9fDTbFjsCzI2uunbLm4EVMXbEPZ/Jap1eFiIiIrml2GCkrK8Px48dx/PhxANeu6VJ3lb/XXnutwaWH586di4sXL2LJkiU4c+YMPvvsM3z66ad45ZVXWuZP0ELUChn+el8ffPHMXfDRqJCuL8O0lfvx6b4s2GxsbiUiImotzd4OPikpCWPHjm10fPbs2fj8888xZ84cXLhwAUlJSfbfJScn4+WXX7Zvevbqq682a9Mzg8EADw8PlJaWOmSfkeIyE179/iR+PlPTPxLTvTM+mDkAOq261T+biIjIWTT1+7tdXJvG0WEEqGmy/eqXbLy19TSqqm3wclPivQf7454+vn/8YiIiImry97dDVtO0RxKJBE8MD8W/F4xEH38trpSb8fyao3h940lUmtncSkRE1FIYRv5AuE6DjbHReGFUVwDAV79k474Ve3HqcqnIlRERETkHhpEmUMll+Mvk3lj77DD4alU4X1iOB1btR0LKeTa3EhER3SGGkWYY2b0zti8ahXv7+qLaKuCdn87iyc9+QX5pldilERERtVsMI83UyU2Jj54Ygndn9IOLQob9GcWYuCwF20/l//GLiYiIqBGGkdsgkUgw664Q/HvhSPQL9EBJRTXmrk3Fn78/gQqzRezyiIiI2hWGkTvQzccd378UjZfGdINEAqw/koP7lu/DiUslYpdGRETUbjCM3CGlXIpXJ/bCV88Ng59WjcyicsxYdQDxSedhZXMrERHRH2IYaSHR3Tpj++IYTO7nB4tNwHvbz+LxTw4ht6RS7NKIiIjaNIaRFuTpqsSHjw3GPx7qD1elDIcyr2DSsr3YeiJP7NKIiIjaLIaRFiaRSPBwZDB+WhiDAcGeKK2sRuy6X/Hf3/6GMhObW4mIiK7HMNJKunR2w3dzozB/bDgkEuDb1EuYsnwvjueUiF0aERFRm8Iw0ooUMileubcn1j8/HIGeLrhYXIEH4w9g5e50NrcSERHVYhhxgGFdvfHTohjc198fVpuAf+78HY8mHMKlqxVil0ZERCQ6hhEH8XBRYMWjg/CvhwfAXSXH4Qs1za2bf8sVuzQiIiJRMYw4kEQiwYzBQfhpYQwGh3jCWGXBwq+PYcmG4zBWVYtdHhERkSgYRkQQ4u2Kb16MwqLx3SGVAD8cu4zJy/ci9eJVsUsjIiJyOIYRkchlUrx8Tw9882IUgjq5IOdKJR7++CCW/ZwOi9UmdnlEREQOwzAissguXvhpUQweGBQIq03A//38Ox5JOIScK2xuJSKijoFhpA3QqhX4v0cGYtmsgdCo5Ei9eBWTlu3FxmOXxC6NiIio1TGMtCHTBgbip0UxiAzthDKTBS9v+A2L1h+Dgc2tRETkxBhG2phgL1esf2E4ltzTAzKpBD8ez8WkpXtx5MIVsUsjIiJqFQwjbZBcJsXC8d3x7dwohHi54nJJJR75+CD+tfMcqtncSkREToZhpA0bHNIJPy2KwYODg2ATgOW7MzDzo4O4WFwudmlEREQthmGkjXNXyfHBwwOw4tFB0KjlOJ5TgsnL9uK71EsQBF7fhoiI2j+GkXZi6oAAbF88CneFeaHcbMUr3/6G+V8fQ2kFm1uJiKh9YxhpRwI9XfD188Px3/f2hFwqwdYTeZi0LAWHMovFLo2IiOi2MYy0MzKpBLFjw/H9S9Ho4u2K3NIqPLr6EN7fcZbNrURE1C4xjLRTA4I9sXVhDB6JDIYgAB/uOY8H4w8gq4jNrURE1L4wjLRjbio53nuoP+IfHwwPFwVOXCrFlOV7seFINptbiYio3WAYcQKT+vlj++IYRHX1RoXZile/P4l5X/2Kkgqz2KURERH9IYYRJ+Hv4YKvnhuG1yb1gkImwbZT+Zi4dC8OZBSJXRoREdEtMYw4EalUghdHd8MPL41A185uyDdU4fFPf0HctjMwW9jcSkREbRPDiBPqF+SBfy8ciceGhUAQgI+TMzEjfj8y9GVil0ZERNQIw4iTclXK8c4D/fDxk0PQyVWBU5cNuG/FXqz7hc2tRETUtjCMOLl7+/ph++JRiOneGVXVNvxl40m88GUqrpSzuZWIiNoGhpEOwFerxhdP34X/mdIbSpkUu04XYOLSFOxNLxS7NCIiIoaRjkIqleC5mK7YGBuNcJ079EYTnvz0MN7692mYLFaxyyMiog7stsLIqlWrEBYWBrVajSFDhmDv3r03fW5SUhIkEkmj29mzZ2+7aLp9fQM8sGX+SDw5PBQA8Mm+LEz/8ADSC4wiV0ZERB1Vs8PIhg0bsHjxYrz++us4duwYYmJiMGnSJGRnZ9/ydefOnUNeXp791r1799sumu6Mi1KG/50egU9nR8LLTYkzeQbct2Ifvjx4gc2tRETkcBKhmd8+w4YNw+DBgxEfH28/1rt3b0yfPh1xcXGNnp+UlISxY8fi6tWr8PT0vK0iDQYDPDw8UFpaCq1We1vvQTemN1bhlW9PIOX3mv6R8b10eO+h/ujsrhK5MiIiau+a+v3drJERs9mM1NRUTJgwocHxCRMm4MCBA7d87aBBg+Dv74/x48djz549t3yuyWSCwWBocKPWodOo8fmcofjb1D5QyqX4z1k9Ji7di6RzerFLIyKiDqJZYaSoqAhWqxW+vr4Njvv6+iI/P/+Gr/H390dCQgK+//57/PDDD+jZsyfGjx+PlJSUm35OXFwcPDw87Lfg4ODmlEnNJJVK8PSIMGyePwI9fN1RVGbCnMQjeHNLGqqq2dxKREStq1nTNLm5uQgMDMSBAwcQFRVlP/7222/jyy+/bHJT6tSpUyGRSLB58+Yb/t5kMsFkMtkfGwwGBAcHc5rGAaqqrXh321l8fuACAKCnrwbLHx2Enn4acQsjIqJ2p1WmaTp37gyZTNZoFESv1zcaLbmV4cOHIz09/aa/V6lU0Gq1DW7kGGqFDG/c3xeJTw9FZ3clzhUYMXXlPny+P4vNrURE1CqaFUaUSiWGDBmCXbt2NTi+a9cuREdHN/l9jh07Bn9//+Z8NDnY2J46bF88CmN7+sBsseGNLacxJ/EICo2mP34xERFRM8ib+4IlS5bgySefRGRkJKKiopCQkIDs7GzMnTsXAPDaa6/h8uXLWLNmDQBg6dKl6NKlC/r27Quz2Yy1a9fi+++/x/fff9+yfxJqcZ3dVfhszlB8eegi3t56Bsm/F2Li0hS8P7M/xvVq+kgYERHRrTQ7jDzyyCMoLi7G3//+d+Tl5SEiIgI//fQTQkNrNtHKy8trsOeI2WzGK6+8gsuXL8PFxQV9+/bF1q1bMXny5Jb7U1CrkUgkeCqqC4Z39cbCr4/hbL4Rz3x+FE9FheIvk3tDrZCJXWKHZLJYoTeYoDeaUGisQoHBBL2xClfKzRgY7InpgwKhkvOfDRG1D83eZ0QM3GekbaiqtuL9Hefw6b4sAEB3nTuWzRqEPgH8Z9JSKs1W6OuFi7rAoTdU1dwba+5LKqpv+T7+Hmq8MKorZg0NgYuSoYSIxNHU72+GEWq2lN8L8V/f/oZCowlKmRR/mtgTz4wIg1QqEbu0NkkQBJSZLLWhon7IqLIfKzBWodBggtFkafL7KmVS+GhU8NWqoNOoodOq4KKUYeOvl6Gv7e3p7K7EsyO74smoULirmj0QSkR0RxhGqFUVl5nw6vcn8fOZAgBATPfO+GDmAOi0apErcxxBEGCotDQcybguXNQdqzA3fb8WF4UMOq0KOs21kKHTqGsea1Xw1db87OGigETSOABWVVvxXeolfJR8HpeuVgIAPFwUmBPdBU+P6AJPV2WLnQMiolthGKFWJwgC1h3Oxv/++zSqqm3o5KrAPx4agHv6tO/mVptNwNUKc+20SL0pEvtUiQkFtT+bLbYmv6+7Sm4PFNeHC596wUOjkt8wZDRXtdWGH4/nYlVSBjILywEAbkoZnogKxXMju8JHwy3/iah1MYyQw2TojVj49XGczqvZtv/xYSH4nyl92lyvgtUmoLj8+qkSU72RDRMKDVUoLDOh2tr0/yw8XBQNQ0b9sKGpHcnQquCqFGeaxGoTsO1UHlbuzsDZ/JqrM6vkUjx6VwheGNUVAZ4uotRFRM6PYYQcymSx4l87f8fHKZkAgK4+blg+axAiAj1a/bOrrTYUlZns4aJu1KKwNnAU1N4Xl5thtTX9X3cvN2VtyLgWLOqHC52mZkSjvawoEgQBu8/qsWJ3Bo7nlAAAFDIJHhwchJfGdEOot5u4BRKR02EYIVHszyjCkm+Oo8BggkImwX/f2xPPjex6W82tJosVhUYTCgy1waKuH6PedEmhsQrF5WY09d9iiaRm/5Rr4UINX60KPtqGIxmd3VVQypu1J2C7IQgCDpwvxord6TiUeQUAIJUA9w8IwLyx4ejhy63/iahlMIyQaK6Wm/HnH05gR1pNc2t0N2/86+GB8POoaW6tW77aOFxU1YaPpi1frU8mlcDH/fqpkmtho+6Yt5sScplzhozbcfTCFazck4Gkc4X2YxP7+mH+uHCHjGoRkXNjGCFRCYKADUdy8OaW06istkKrlsNHo4LeaIKxqvnLV28ULny010YyvFyVXFp8B05dLsWHezKw7dS1606N7uGDBePCEdnFS8TKiKg9YxihNiGzsAyL1h/HyculDY6rFVJ7k6d9NYn2upEMjQqerjdevkqtI73AiFVJ5/Hj8cuoa68ZFuaFBeO6Y0S4N/9ZEFGzMIxQm2G22HAosxhyqaQmcGjVLbZ8lVrHxeJyfJR8Ht+lXrKvLBoQ7IkFY8MxvreO/+yIqEkYRojojuWWVCIhJRNfH86GqXZPlV5+GsSODcfkfv6QcWqMiG6BYYSIWkyh0YRP92Xhy4MXUF67m2zXzm54aUw3TB8UCAWbgonoBhhGiKjFlVSY8fmBC0jcfwGllTWrnQI9XTB3TDfMHBLUbvZcISLHYBgholZTZrLgq0MXsXpvJorKzAAAnUaFF0Z1xWPDQkTbbZaI2haGESJqdVXVVmw4koOPks8jr7QKANDJVYFnR4bhyagu8HBRiFwhEYmJYYSIHMZssWHjsUtYlXQeF4srAAAalRyzo7vgmZFh8HLjlYKJOiKGESJyOIvVhq0nay7Kl64vAwC4KGR4bFjNRfl8tWqRKyQiR2IYISLR2GwCdp4uwMo96Th1ueZqzkqZFDMjgzB3dDcEe7mKXCEROQLDCBGJThAEJP9eiJW7M3D04lUANdcRmj4wEPPGdkM3H3eRKySi1sQwQkRthiAI+CXrCj7ck4G96UUAaq6gPLmfP2LHhKNPAP+7JnJGDCNE1CYdzynByt0Z+PlMgf3Y3b11iB0bjkEhnUSsjIhaGsMIEbVpZ/IM+HBPBraezEPd/4VGhndG7NhwDO/qxevfEDkBhhEiahfOF5YhPuk8Nh27DEvtpYKHhHbC/HHhGNPDh6GEqB1jGCGidiXnSgU+TjmPb45egrn2onwRgVrMHxuOCX38IOVF+YhajSAIEAS0+H9nDCNE1C4VGKqwOiUTX/2Sjcrqmovydde5I3ZsOO7r7w85L8pH1GQ2m4CrFWbojSYUGKqgN5pQaDRBb6hCgcEEvbHmmN5owsdPDsHYnroW/XyGESJq166Um5G4Pwuf778Ao8kCAAjxcsW8Md0wY3AQlHKGEuq4rDYBxWWm2iBRBb3B1Chc6A1VKDSa7NOff+S9B/vhkaEhLVonwwgROQVDVTW+PHgRn+zNxNWKmisF+3uo8eKorph1VwivFExOpdpqQ1GZqTZcXAsWhcZ6IxkGE4rKTGhixgAAeLkpodOooNOqa+41KvjW/axVQadRQ6dVQSVv2f+eGEaIyKlUmC1Y90s2ElIyoTeaAACd3ZV4LqYrnhgeCncVrxRMbZfJYoXecC1Y2KdNDKYGIxlXKsxo6reyVAJ4u6sahQuf2vu6x53dVaKNJDKMEJFTqqq24rvUS/go+TwuXa0EAHi4KDAnugueHtEFnq68KB85TqXZap8aaRgu6n6u+V1J7aheU8ilEvjUBgyf2hEL39r7muBR87O3m7LN91AxjBCRU6u22vDj8VysSspAZmE5AMBNKcMTUaF4bmRX+GhUIldI7VmZyVIvXNT0XtinTepNl9T1MzWFUiatCRk3CRd1916uSqdZPcYwQkQdgtUmYNupmisFn803AgBUcikevavmSsEBni4iV0hthSAIMFRaoDdet5LEYEKBsQqF9Y5VmK1Nfl+1Qnqt/0Kjhs+N+jE0Kni6KjrcvjkMI0TUoQiCgN1n9VixOwPHc0oAAAqZBA8ODsJLY7oh1NtN3AKp1dRfvlrXe9HgvnZUo9Bogql2D5umcFfJa6dKbtzsWXevUck7XMhoKoYRIuqQBEHAgfPFWLE7HYcyrwCoafS7f0AA5o0NRw9fjcgVUlNZbQKKyxtOi9T1YxTUNYMaqlBYZkK1telfZVq1vCZc1Bu1qL/KpO5nNzZF3zGGESLq8I5euIKVezKQdK7QfmxiXz/MHxeOiEAPESvr2OovX60fLgrrBY4CQxWKy82wNmP9at3y1QYjGbXhwld7bQqFy8Edh2GEiKjWyUul+HBPBran5duPje7hgwXjwhHZxUvEypyLyWKt2d3TeP1oRt1qk5rAUVze9OWrEgng7aaqDRP1pkiuCxs+Ii5fpZtjGCEius7vBUas2pOBzb/l2jeMGhbmhQXjumNEuDfn/W+i/vLVhuGiqnZr8ZpjV5uxfFUmlcDHXXVtRUm9BlD7HhntZPkq3RzDCBHRTVwoKsdHyefx/a+X7L0GA4I9sWBsOMb31nWYUFJmsjRo8ry+8bNuKaux6vaWr14fLnzqHfN2c57lq3RzDCNERH8gt6QSCSmZ+Ppwtn2VRS8/DWLHhmNyP3/I2uGXpSAIMFRZGm0ffqMLpZU3c/nqtWbP61aU1OvR6IjLV+nmGEaIiJqo0GjCp/uy8OXBC/Yv6K6d3fDSmG6YPigQijYwTSAIAq5WVDcKF4X1L5RWe9+c5atuStm1KZKbXLfER6OGVs3lq9R8rRpGVq1ahffffx95eXno27cvli5dipiYmJs+Pzk5GUuWLEFaWhoCAgLwpz/9CXPnzm3y5zGMEJEjlFSY8fmBC0jcfwGllTX9D4GeLpg7phtmDglqlVUYNpuA4nJzw2bPG4xkFBpNMFubHjK0ark9XNivWVK3soTLV8lBWi2MbNiwAU8++SRWrVqFESNG4OOPP8Ynn3yC06dPIySk8aWHs7KyEBERgeeffx4vvvgi9u/fj3nz5uHrr7/Ggw8+2KJ/GCKillBmsmDtoZorBReVmQEAOo0KL4zqiseGhcBV+cdf4BarDUVl5puEi2s9GUVlzVu+2slVUdN/UW9lie/1DaBaLl+ltqHVwsiwYcMwePBgxMfH24/17t0b06dPR1xcXKPnv/rqq9i8eTPOnDljPzZ37lz89ttvOHjwYJM+k2GEiMRQVW3F+sPZ+DglE3mlVQBqwsCzI8MQHd7Z3ntRf0vxuuBRXG5q9vLVummR+tct8dHU7pHB5avUDjX1+7tZ43Nmsxmpqan485//3OD4hAkTcODAgRu+5uDBg5gwYUKDY/feey8+/fRTVFdXQ6FQNHqNyWSCyWRq8IchInI0tUKGOSPC8NiwUPzw6yXEJ5/HxeIK/HPn78DO3//w9TKpBJ3dlfWmSdSN+jF0GjU6u3P5KnVszQojRUVFsFqt8PX1bXDc19cX+fn5N3xNfn7+DZ9vsVhQVFQEf3//Rq+Ji4vDm2++2ZzSiIhajVIuxay7QvDQkCBsPZmHT/Zm4Uq52X6Z95tdt8TLTdkuV+QQOdptdS5d31EtCMItu6xv9PwbHa/z2muvYcmSJfbHBoMBwcHBt1MqEVGLkcukmDYwENMGBopdCpFTaVYY6dy5M2QyWaNREL1e32j0o46fn98Nny+Xy+Ht7X3D16hUKqhUquaURkRERO1UsyYplUolhgwZgl27djU4vmvXLkRHR9/wNVFRUY2ev3PnTkRGRt6wX4SIiIg6lmZP0yxZsgRPPvkkIiMjERUVhYSEBGRnZ9v3DXnttddw+fJlrFmzBkDNypmVK1diyZIleP7553Hw4EF8+umn+Prrr5v8mXXTOmxkJSIiaj/qvrf/cOGucBs+/PBDITQ0VFAqlcLgwYOF5ORk++9mz54tjB49usHzk5KShEGDBglKpVLo0qWLEB8f36zPy8nJEQDwxhtvvPHGG2/t8JaTk3PL7/l2sR28zWZDbm4uNBpNi25HXNcYm5OTw/1LWhnPtWPwPDsGz7Nj8Dw7RmueZ0EQYDQaERAQAKn05p0h7WIfYKlUiqCgoFZ7f61Wy3/RHYTn2jF4nh2D59kxeJ4do7XOs4eHxx8+h7vsEBERkagYRoiIiEhUHTqMqFQq/O1vf+OeJg7Ac+0YPM+OwfPsGDzPjtEWznO7aGAlIiIi59WhR0aIiIhIfAwjREREJCqGESIiIhIVwwgRERGJqsOGkZSUFEydOhUBAQGQSCTYtGmT2CU5nbi4OAwdOhQajQY6nQ7Tp0/HuXPnxC7L6cTHx6N///72DYuioqKwbds2sctyenFxcZBIJFi8eLHYpTidN954AxKJpMHNz89P7LKc0uXLl/HEE0/A29sbrq6uGDhwIFJTUx1eR4cNI+Xl5RgwYABWrlwpdilOKzk5GbGxsTh06BB27doFi8WCCRMmoLy8XOzSnEpQUBDeffddHD16FEePHsW4ceMwbdo0pKWliV2a0zpy5AgSEhLQv39/sUtxWn379kVeXp79dvLkSbFLcjpXr17FiBEjoFAosG3bNpw+fRoffPABPD09HV5Lu9gOvjVMmjQJkyZNErsMp7Z9+/YGjxMTE6HT6ZCamopRo0aJVJXzmTp1aoPHb7/9NuLj43Ho0CH07dtXpKqcV1lZGR5//HGsXr0ab731ltjlOC25XM7RkFb23nvvITg4GImJifZjXbp0EaWWDjsyQo5XWloKAPDy8hK5EudltVqxfv16lJeXIyoqSuxynFJsbCymTJmCu+++W+xSnFp6ejoCAgIQFhaGWbNmITMzU+ySnM7mzZsRGRmJmTNnQqfTYdCgQVi9erUotTCMkEMIgoAlS5Zg5MiRiIiIELscp3Py5Em4u7tDpVJh7ty52LhxI/r06SN2WU5n/fr1+PXXXxEXFyd2KU5t2LBhWLNmDXbs2IHVq1cjPz8f0dHRKC4uFrs0p5KZmYn4+Hh0794dO3bswNy5c7Fw4UKsWbPG4bV02Gkacqz58+fjxIkT2Ldvn9ilOKWePXvi+PHjKCkpwffff4/Zs2cjOTmZgaQF5eTkYNGiRdi5cyfUarXY5Ti1+lPo/fr1Q1RUFLp164YvvvgCS5YsEbEy52Kz2RAZGYl33nkHADBo0CCkpaUhPj4eTz31lENr4cgItboFCxZg8+bN2LNnD4KCgsQuxykplUqEh4cjMjIScXFxGDBgAJYtWyZ2WU4lNTUVer0eQ4YMgVwuh1wuR3JyMpYvXw65XA6r1Sp2iU7Lzc0N/fr1Q3p6utilOBV/f/9Gf2Hp3bs3srOzHV4LR0ao1QiCgAULFmDjxo1ISkpCWFiY2CV1GIIgwGQyiV2GUxk/fnyjFR1PP/00evXqhVdffRUymUykypyfyWTCmTNnEBMTI3YpTmXEiBGNtlv4/fffERoa6vBaOmwYKSsrQ0ZGhv1xVlYWjh8/Di8vL4SEhIhYmfOIjY3FunXr8OOPP0Kj0SA/Px8A4OHhARcXF5Grcx5/+ctfMGnSJAQHB8NoNGL9+vVISkpqtJqJ7oxGo2nU7+Tm5gZvb2/2QbWwV155BVOnTkVISAj0ej3eeustGAwGzJ49W+zSnMrLL7+M6OhovPPOO3j44Ydx+PBhJCQkICEhwfHFCB3Unj17BACNbrNnzxa7NKdxo/MLQEhMTBS7NKfyzDPPCKGhoYJSqRR8fHyE8ePHCzt37hS7rA5h9OjRwqJFi8Quw+k88sgjgr+/v6BQKISAgABhxowZQlpamthlOaUtW7YIERERgkqlEnr16iUkJCSIUodEEATB8RGoeWw2G3Jzc6HRaCCRSMQuh4iIiJpAEAQYjUYEBARAKr15m2qrT9PExcXhhx9+wNmzZ+Hi4oLo6Gi899576NmzZ5PfIzc3F8HBwa1YJREREbWWnJycWy5gaPUwUrcl+NChQ2GxWPD6669jwoQJOH36NNzc3Jr0HhqNBkDNH0ar1bZmuURERNRCDAYDgoOD7d/jN+PwaZrCwkLodDokJyc3eUtwg8EADw8PlJaWMowQERG1E039/nb4PiNN2RLcZDLBYDA0uLWGU5dLMe+rVFwpN7fK+xMREdEfc2gYEZq4JXhcXBw8PDzst9boFxEEAf/93Qn8dDIfE5emIOX3whb/DCIiIvpjDp2miY2NxdatW7Fv375bNrKYTKYGGzbVzTm19DTN6VwDFq0/hnR9GQDgmRFh+NPEnlAruHkRERHRnWrqNI3DwsiCBQuwadMmpKSkNHsnztbsGamqtiLupzP44uBFAEAvPw2WzRqEnn63brYhIiKiW2szPSOCIGD+/Pn44YcfsHv37ja3JbhaIcOb0yKQOGcoOrsrcTbfiKkr9+Hz/VloB1uwEBERtXutHkZiY2Oxdu1arFu3zr4leH5+PiorK1v7o5tlbC8dti0ahbE9fWC22PDGltOYk3gEemOV2KURERE5tVafprnZjqmJiYmYM2dOk97DkUt7BUHA2kMX8dbWMzBZbPB2U+IfD/XH+N6+rfq5REREzqbN9YzcCTH2GUkvMGLh+uM4k1ezrPiJ4SF4fXIfuCjZ3EpERNQUbaZnpL3q7qvBpthoPB9T0+Oy9lA27luxF6cul4pcGRERkXNhGLkFlVyG16f0wdpnh8FXq8L5wnI8sGo/ElLOw2Zr8wNKRERE7QLDSBOM7N4Z2xeNwr19fVFtFfDOT2fxxKe/IK+0bTXhEhERtUcMI03UyU2Jj54Ygvce7AcXhQwHzhdj4tK92HYyT+zSiIiI2jWGkWaQSCR4ZGgIfloUgwFBHiitrMZLX/2KP333G8pNFrHLIyIiapcYRm5DWGc3fPdSNOaPDYdEAnxz9BImL9+LY9lXxS6NiIio3WEYuU0KmRSv3NsT658fjkBPF1wsrsBDHx3Eiv+kw8rmViIioiZjGLlDw7p646dFMbh/QACsNgEf7PodsxIOIudKhdilERERtQsMIy3Aw0WB5Y8OwtJHBsJdJceRC1cxedlebDp2WezSiIiI2jyGkRY0fVAgti2KQWRoJxhNFizecByL1h+Doapa7NKIiIjaLIaRFhbs5Yr1LwzHknt6QCaV4MfjuZi0dC8OZ10RuzQiIqI2iWGkFchlUiwc3x3fzo1CqLcrLpdUYlbCQfxzxzlUW21il0dERNSmMIy0osEhnbB1YQxmDgmCTQBW7snAQ/EHkFVULnZpREREbQbDSCtzV8nx/swB+PCxwfBwUeC3S6WYsnwvNhzJRju4YDIREVGrYxhxkCn9/bF9cQyiunqjwmzFq9+fxEtrf8XVcrPYpREREYmKYcSB/D1c8NVzw/DapF5QyCTYnpaPictSsD+jSOzSiIiIRMMw4mBSqQQvju6GjfNGoKuPGwoMJjz+yS9456czMFmsYpdHRETkcAwjIokI9MDWBTF4fFgIACAhJRMPfHgAGXqjyJURERE5FsOIiFyUMrz9QD988lQkvNyUOJ1nwJTl+/DlwQtsbiUiog6DYaQNuLuPL7YvjsGoHj4wWWz4649pePaLoygqM4ldGhERUatjGGkjdBo1Pp8zFH+b2gdKuRS7z+oxcWkK9pzTi10aERFRq2IYaUOkUgmeHhGGzfNHoJefBkVlZjydeAR/+/EUqqrZ3EpERM6JYaQN6uWnxabYEXhmRBgA4IuDFzF1xT6czjWIXBkREVHLYxhpo9QKGf7f1D744pm74KNRIV1fhukf7scnezNhs7G5lYiInAfDSBs3uocPti+Kwd29fWG22vDW1jOYnXgYBYYqsUsjIiJqEQwj7YC3uwqrnxqCtx+IgFohxd70IkxcmoLtp/LFLo2IiOiOMYy0ExKJBI8PC8W/F8QgIlCLqxXVmLs2Fa/9cAIVZovY5REREd02hpF2Jlznjh9eGoG5o7tBIgG+PpyD+5bvw4lLJWKXRkREdFsYRtohpVyKP0/qhXXPDYe/hxqZReWYseoAPtyTASubW4mIqJ1hGGnHorp5Y/uiUZjSzx8Wm4D3d5zDo6sP4XJJpdilERERNRnDSDvn4arAyscG4Z8zB8BNKcPhrCuYuDQFm3/LFbs0IiKiJmEYcQISiQQPDQnCT4tiMCjEE8YqCxZ+fQxLNhyHsapa7PKIiIhuiWHEiYR6u+HbF6OwaHx3SCXAD8cuY/LyvUi9eEXs0oiIiG6KYcTJyGVSvHxPD3zzYhSCOrkg50olZn50EP/a9TssVpvY5RERETXCMOKkIrt4YduiGMwYFAibACz/TzpmfnwQF4vLxS6NiIioAYYRJ6ZRK/CvRwZi+aODoFHLcSy7BJOX7cV3qZcgCFwCTEREbQPDSAdw/4AAbF88CneFeaHcbMUr3/6G+euOobSCza1ERCQ+hpEOItDTBV8/Pxz/fW9PyKUSbD2Zh4nLUnDwfLHYpRERUQfHMNKByKQSxI4Nxw/zohHW2Q15pVV47JNDeHfbWZgtbG4lIiJxMIx0QP2DPPHvBSPx6F3BEATgo+TzmBG/H+cLy8QujYiIOiCGkQ7KTSVH3Iz++OiJIfB0VeDUZQOmLN+Lr365yOZWIiJyKIaRDm5ihB92LB6FkeGdUVVtw+sbT+H5NakoLjOJXRoREXUQDCMEX60aa565C/8zpTeUMil+PlOAicv2Ivn3QrFLIyKiDoBhhAAAUqkEz8V0xabYEeiuc0eh0YTZnx3G37ecRlW1VezyiIjIiTGMUAN9ArTYsmAkZkeFAgA+25+F6R/ux9l8g8iVERGRs2IYoUbUChnenBaBxDlD0dldibP5Rty/cj8S92exuZWIiFocwwjd1NheOmxfPArjeulgttjw5pbTmJN4BHpjldilERGRE5EI7eCvugaDAR4eHigtLYVWqxW7nA5HEASsPXQRb209A5PFBi83Jd57sD/u6eMrdmkdliAIKK2sht5oQoGhCgWGmvsr5WYMDPbE5H7+kEklYpdJRB1cU7+/GUaoydILjFi4/jjO5NX0jzw+LAT/M6UPXJQykStzHoIgwGiyQF8vYNQFDn3t4wJjze9utWtu185umDc2HNMGBkAh4wAoEYmDYYRahclixQc7f0dCSiYAoKuPG5bPGoSIQA+RK2v7ykwWe6jQG6sajGhcO2ZCZTNWL3m6KuCrUUOnVcFXq4arUobNv+WipPYiiMFeLnhpdDgeHBIIlZyhkYgci2GEWtW+9CL817fHUWAwQSGT4L8m9MQLMV0h7YBTAxVmS71RC1PtqEbjEY1yc9NDhlYth69WDV9tTdDQadTwrQ0cvrWPfTQqqBWNA0a5yYK1hy5i9d5MFJWZAQD+Hmq8OKorZt0VcsPXEBG1BoYRanVXy8348w8nsCOtAAAQ1dUbHzw8AAGeLiJX1jKqqq01QaJ2FKPuZ3vwqD1mNFma/J4alRw+WhV864ULXb2AUXffElNflWYr1h/JxsfJmcg31DQdd3ZX4YVRYXh8WCjcVPI7/gwiolthGCGHEAQB3xzNwRubT6Oy2goPFwXeeaAfpvT3F7u0mzJZrCg0mlBguDaKUWBsPF1SWlnd5Pd0Ucjg56GGTlMbMOru7aMZNcfECAAmixXfpV7Cqj3ncbmkEgDQyVWBZ0eG4anoLtCqFQ6viYg6BoYRcqisonIsXn8Mv10qBQA8NCQIb9zfF+4O/PKtttpqQ0ZNmNDXH8WoN31ytaLpIUMll16bGtGqrxvRuDaa4a6SQyJp21NU1VYbNh67jFV7MnChuAIAoFHL8XR0FzwzMgyerkqRKyQiZ9PmwsiqVavw/vvvIy8vD3379sXSpUsRExPTpNcyjLQP1VYblv2cjg+TMiAIQIiXK5bOGojBIZ3u6H0tVhuKyszX9WHUNn/WjmIUGqvs/RFNoZRJ641a1IQKnX365Fr40KrbfshoLovVhq0n87BydwbS9WUAADelDE9EheL5mK7o7K4SuUIichZtKoxs2LABTz75JFatWoURI0bg448/xieffILTp08jJCTkD1/PMNK+/JJZjCXf/IbLJZWQSSVYOK47Ysd2g/y6JaZWm4DiclO9Hoy6paxV9Xo1TCgqM6Gp/5YqZBJ7c6fvdVMk9mkTjRqergqnCxnNZbMJ2JGWjxW7M3C6drm2WiHFY3eF4oVRXeHnoRa5QiJq79pUGBk2bBgGDx6M+Ph4+7HevXtj+vTpiIuL+8PXM4y0P6WV1fjrplPY/FsuAGBQiCf6+Gvt0ycFhpqRDKutaf/6yaQS+Lirrk2X1IYK3XXTJ51clR1yRc+dEAQBu8/qsXx3Bn7LKQFQM3L08NAgzB3dDUGdXMUtkIjarTYTRsxmM1xdXfHtt9/igQcesB9ftGgRjh8/juTk5EavMZlMMJlM9scGgwHBwcEMI+3QpmOX8T+bTqHsJitOpJKaFR51oxf2oHHd9Im3m4o7irYyQRCwL6MIK/6TgcMXrgAA5FIJZgwOxLwx4ejS2U3kComovWlqGGn17sKioiJYrVb4+jbcOtzX1xf5+fk3fE1cXBzefPPN1i6NHGD6oEAMCe2Erw9nQy6T2qdL6gKHt5uy0fQNiUMikSCmuw9iuvvgl8xirNidgX0ZRfjm6CV8l3oJ9w8IwPxx4QjXacQulYicjMOWOlw/Py8Iwk3n7F977TUsWbLE/rhuZITap2AvV/xpYi+xy6BmGNbVG8O6euPX7KtYuTsDu8/qsel4Ln78LReTIvwwf2x39AngKCURtYxWDyOdO3eGTCZrNAqi1+sbjZbUUalUUKnY0U8ktsEhnfDZnKE4dbkUK3dnYHtaPn46WXO7u7cO88d1x8BgT7HLJKJ2rtXHx5VKJYYMGYJdu3Y1OL5r1y5ER0e39scTUQuICPTAR08OwY7Fo3D/gABIJcDPZ/SY/uF+PPnpLzhS22NCRHQ7HLq096OPPkJUVBQSEhKwevVqpKWlITQ09A9fz9U0RG1LZmEZViWdx8Zjl+0rooaFeWHh+O6I7ubd4ZdNE1GNNrOaps6qVavwj3/8A3l5eYiIiMD//d//YdSoUU16LcMIUduUc6UCq5LO47vUHFRba/5XMjjEEwvGdceYnj4MJUQdXJsLI3eCYYSobcsrrcTHyZn4+nA2TBYbACAiUIv5Y7tjQh9f7v1C1EExjBCRw+mNVfhkbxbWHrqICrMVANDTV4P548IxuZ8/94oh6mAYRohINFfKzfhsXxa+OHABxtoN77r6uCF2TDimDQzg3jJEHQTDCBGJrrSyGl8cuIBP92WhtLLmasnBXi6YNyYcDw4OglLOUELkzBhGiKjNKDNZ8OXBi/hkbyaKy2uuruzvocbc0d3wyNBgqBUykSskotbAMEJEbU6l2Yp1h7PxcfJ56I0115/y0ajwQkxXPD48BK5Kh20KTUQOwDBCRG1WVbUV36ZewkdJ53G5pBIA0MlVgediuuKpqFBo1AqRKySilsAwQkRtntliw6Zjl/FhUgYuFlcAALRqOeaMCMMzI7rA01UpcoVEdCcYRoio3bBYbfj3iTys3JOBDH0ZAMBNKcOTUV3wXEwYOrvzWlVEre1WF7C9XQwjRNTu2GwCtqflY8XuDJzJMwAA1AopHh8WihdGdYWvVi1yhUTtT1W1FYVGEwoMVSgwmKA31t4bqlBQ+3OBoQrLHx2EsT11LfrZTf3+ZrcYEbUZUqkEk/v5Y1KEH34+o8eK3ek4cakUn+7LwpeHLuKRyGC8OLorgjq5il0qkejMFhv0xirojbXBwtAwcOgNJhQYq1BSUd2k9ys0mFq54pvjyAgRtVmCICAlvQgr/pOOoxevAgDkUgkeHByEeWO7IdTbTeQKiVpetdWGojKTPVzorxvRKDDUBJArtcvkm0Ill8JXq4avVgWdVg1fTd3PKvhq1NBp1Qj0dIGLsmWX2XOahoichiAIOJR5BSt2p+PA+WIAgFQCTBsYiNix3RCu04hcIdEfs1htKC431wYMk32KpCZs1AUOE4rLTWjqN7NSJq0JFHVBQ6O2B4z64UOrloty4UqGESJySqkXr2DF7gwknSsEAEgkwOQIf8wfF47e/vz/AzmezSZcCxnGhtMlhfUeF5WZYGviN65cKoGvVg0fjQq+9rChhk5T83Nd4PB0VbTpq2MzjBCRUzt5qRQrdqdj5+kC+7F7+vhiwbhw9A/yFK8wcho2m4CrFeaaMGGsQmFdyGgwomFCYZkJ1iamDJlUAh931bXpktrRjOunTzq5Kp3iatcMI0TUIZzJM+DDPRnYejLPPrQ9uocPFowLR2QXL3GLozZJEASUVFRDb19hUtXoZ31tj0a1tWlfkVIJ0NldZR+9qAsa10+feLupOtTVqxlGiKhDydCXYVVSBn48nmv/W+rwrl5YOK47orp5t+mhbGoZgiDAUGVpuLKkdlXJ9c2fZoutye/b2V1pH72omSJpOKLhq1XD203Jq1HfAMMIEXVIF4vLEZ90Ht//esn+t9ohoZ0wf1w4xvTwYShphwRBQJnJ0nDJal3gMDZcbVJV3fSQ4eWmvNaDoVE1XG1S+3NndxUUDBm3jWGEiDq0yyWV+Dj5PNYfybH/LbhfoAfmjwvHPb19nWI+3hmUmywNp0jqjV7Uv68wW5v8np6uinoho+F0iY+m7l4FlZxXi25tDCNERAD0hiokpGTiq1+yUVld84XWy0+D+ePCMSnCv0PN3ztSpdnaaGqk/hLWuumTMpOlye+pUcvtoaJub4z6Ixp1q0/UCoaMtoJhhIionuIyEz7dl4U1By/avwC7+bghdmw47h8QwPn+Jqq/tbi+/hbj9fozCgxVMFQ1PWS4q+TQaVX1goW60fSJTquCq5Kbhrc3DCNERDdQUmHG5wcu4LN9WfYvzBAvV8wb0w0zBgdBKe+YocRssaGwrOGOn3eytTgAuChkDXsw6gWL+stZ3VUMGc6KYYSI6BaMVdX48tBFfLI3y76tdoCHGnPHdMPDkcFOM9R/o63F649otMTW4g2mSjTXVpu4q8TZ9ZPaDoYRIqImqDBbsO6XbCSkZEJvrLlQmI9GhRdHdcVjw0La7NRAa24t3mC6xB4wao9p1NC6MGRQ0zCMEBE1Q1W1Fd8ezUF80nnkllYBqFn6+ezIMDwVFQqNWuGQOlpra/HrN+K69vjaiEZb31qc2h+GESKi22C22PDDr5ewKuk8sq9UAAA8XBR4ekQXPB0dBg/X2wsl128trr/hiMbtby3uc90S1rqtxXVaFbycZGtxan8YRoiI7oDFasPm33Kxck8GMgvLAdSs+ngyKhTPjQyDt7sKwLWtxeuvJKm/b0bNiEbztxb3rg0Z9Xswrl9d0tG2Fqf2h2GEiKgFWG0Ctp3Kw8rdGTibbwRQs0qkl7+mJmQYTDBb72Br8eunS7i1ODkRhhEiohZkswn4+UwBVuzOwMnLpY1+X7e1uK7eEtb6y1p1mppdP7m1OHUkTf3+bptt4kREbYxUKsGEvn64p48vDmddwZVys336hFuLE90ZhhEiomaQSCQY1tVb7DKInArHC4mIiEhU7WJkpK6txWAwiFwJERERNVXd9/Yftae2izBiNNZ0sAcHB4tcCRERETWX0WiEh4fHTX/fLlbT2Gw25ObmQqPRtOjugAaDAcHBwcjJyeEqnVbGc+0YPM+OwfPsGDzPjtGa51kQBBiNRgQEBEAqvXlnSLsYGZFKpQgKCmq199dqtfwX3UF4rh2D59kxeJ4dg+fZMVrrPN9qRKQOG1iJiIhIVAwjREREJKoOHUZUKhX+9re/QaVSiV2K0+O5dgyeZ8fgeXYMnmfHaAvnuV00sBIREZHz6tAjI0RERCQ+hhEiIiISFcMIERERiYphhIiIiETVYcNISkoKpk6dioCAAEgkEmzatEnskpxOXFwchg4dCo1GA51Oh+nTp+PcuXNil+V04uPj0b9/f/uGRVFRUdi2bZvYZTm9uLg4SCQSLF68WOxSnM4bb7wBiUTS4Obn5yd2WU7p8uXLeOKJJ+Dt7Q1XV1cMHDgQqampDq+jw4aR8vJyDBgwACtXrhS7FKeVnJyM2NhYHDp0CLt27YLFYsGECRNQXl4udmlOJSgoCO+++y6OHj2Ko0ePYty4cZg2bRrS0tLELs1pHTlyBAkJCejfv7/YpTitvn37Ii8vz347efKk2CU5natXr2LEiBFQKBTYtm0bTp8+jQ8++ACenp4Or6VdbAffGiZNmoRJkyaJXYZT2759e4PHiYmJ0Ol0SE1NxahRo0SqyvlMnTq1weO3334b8fHxOHToEPr27StSVc6rrKwMjz/+OFavXo233npL7HKcllwu52hIK3vvvfcQHByMxMRE+7EuXbqIUkuHHRkhxystLQUAeHl5iVyJ87JarVi/fj3Ky8sRFRUldjlOKTY2FlOmTMHdd98tdilOLT09HQEBAQgLC8OsWbOQmZkpdklOZ/PmzYiMjMTMmTOh0+kwaNAgrF69WpRaGEbIIQRBwJIlSzBy5EhERESIXY7TOXnyJNzd3aFSqTB37lxs3LgRffr0Ebssp7N+/Xr8+uuviIuLE7sUpzZs2DCsWbMGO3bswOrVq5Gfn4/o6GgUFxeLXZpTyczMRHx8PLp3744dO3Zg7ty5WLhwIdasWePwWjrsNA051vz583HixAns27dP7FKcUs+ePXH8+HGUlJTg+++/x+zZs5GcnMxA0oJycnKwaNEi7Ny5E2q1WuxynFr9KfR+/fohKioK3bp1wxdffIElS5aIWJlzsdlsiIyMxDvvvAMAGDRoENLS0hAfH4+nnnrKobVwZIRa3YIFC7B582bs2bMHQUFBYpfjlJRKJcLDwxEZGYm4uDgMGDAAy5YtE7ssp5Kamgq9Xo8hQ4ZALpdDLpcjOTkZy5cvh1wuh9VqFbtEp+Xm5oZ+/fohPT1d7FKcir+/f6O/sPTu3RvZ2dkOr4UjI9RqBEHAggULsHHjRiQlJSEsLEzskjoMQRBgMpnELsOpjB8/vtGKjqeffhq9evXCq6++CplMJlJlzs9kMuHMmTOIiYkRuxSnMmLEiEbbLfz+++8IDQ11eC0dNoyUlZUhIyPD/jgrKwvHjx+Hl5cXQkJCRKzMecTGxmLdunX48ccfodFokJ+fDwDw8PCAi4uLyNU5j7/85S+YNGkSgoODYTQasX79eiQlJTVazUR3RqPRNOp3cnNzg7e3N/ugWtgrr7yCqVOnIiQkBHq9Hm+99RYMBgNmz54tdmlO5eWXX0Z0dDTeeecdPPzwwzh8+DASEhKQkJDg+GKEDmrPnj0CgEa32bNni12a07jR+QUgJCYmil2aU3nmmWeE0NBQQalUCj4+PsL48eOFnTt3il1WhzB69Ghh0aJFYpfhdB555BHB399fUCgUQkBAgDBjxgwhLS1N7LKc0pYtW4SIiAhBpVIJvXr1EhISEkSpQyIIguD4CERERERUgw2sREREJCqGESIiIhIVwwgRERGJimGEiIiIRMUwQkRERKJiGCEiIiJRMYwQERGRqBhGiIiISFQMI0RERCQqhhEiIiISFcMIERERiYphhIiIiET1/wFOAT1WMrOt0AAAAABJRU5ErkJggg==", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "import numpy as np\n", "import matplotlib.pyplot as plt\n", "\n", "x=[1,2,3,4,5,6]\n", "valores = np.array([85,35,42,8,15,22])\n", "\n", "valores_escalados = np.array([1,0.35,0.44,0,0.09,0.18])\n", "\n", "valores_estandarizados = np.array([1.99,0.02,0.3,-1.05,-0.77,-0.49])\n", "\n", "fig, axs = plt.subplots(3)\n", "axs[0].plot(x, valores)\n", "axs[1].plot(x, valores_escalados)\n", "axs[2].plot(x, valores_estandarizados)\n", "\n" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "## Fase 2: División del Conjunto de Datos" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Conjunto de entrenamiento, testeo, [y validación]\n", "\n", "\n", "- Debemos asegurar, para evitar el sobreajuste (_overfitting_), que la evaluación del modelo se realice en un conjunto de datos distinto a aquel sobre el cual se entrenó. \n", "\n", "- Sobreajuste: nuestro modelo tiene buen rendimiento sobre el dataset de entrenamiento, pero sus resultados son muy inferiores cuando se encuentra a datos no vistos previamente. Causa probable: el modelo está memorizando los datos de entrenamiento, sin poder generalizar. \n", "\n", "- Si evaluamos sobre el mismo conjunto de datos sobre el que entrenamos, no podemos saber si estamos sobreajustando. **Sobreajustar es el peligro mayor cuando hacemos aprendizaje automático**" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Conjunto de entrenamiento, testeo, [y validación]\n", "\n", "\"Drawing\"\n", "\n", "\n", "[Ejemplo COVID-19 en Uruguay](https://github.com/gmonce/datascience/blob/master/src/Sobreajustando.ipynb)" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Conjunto de entrenamiento, testeo, [y validación]\n", "\n", "\"Drawing\"\n", "\n" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Conjunto de entrenamiento, testeo, [y validación]\n", "\n", "\"Drawing\"\n", "\n", "\n", "- Separación inicial: conjunto de entrenamiento y de evalución. \n", "\n", "- Cuantas más instancias para entrenar tengamos, probablemente mejor será nuestro modelo, PERO...\n", "\n", "- Cuantas más intancias para evaluar tengamos, menor será la varianza de nuestros resultados.\n", "\n", "- Usualmente se divide como 80%-20%, o 70%-30%\n", "\n", "Fuente de la imagen: [About Train, Validation and Test Sets in Machine Learning](https://towardsdatascience.com/train-validation-and-test-sets-72cb40cba9e7) - Tarang Shah" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Conjunto de validación\n", "\n", "- Si queremos ajustar los parámetros del modelo (lo veremos en breve), no es conveniente hacerlo en el conjunto de evaluación (ya que podríamos estar sobreajustando, nuevamente). \n", "\n", "- Opción 1: separar una parte del conjunto de entrenamiento para utilizarlo en esa etapa\n", "\n", "- Opción 2: utilizar validación cruzada (cross-validation)" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Estratificación\n", "\n", "- Aunque no queremos que la evaluación se realice en el conjunto de entrenamiento, sí nos interesa que la distribución de los ejemplos en uno y otro sea similar.\n", "\n", "- Al hacer la división de los conjuntos, lo usual es elegir las instancias al azar, para evitar que las agrupaciones u ordenamientos presentes en el conjunto original puedan dar lugar a distribuciones distintas. \n", "\n", "- Un paso más, especialmente importante cuando las clases objetivo están _desbalanceadas_ (es decir, hay muchos más ejemplos de una clase que de otras), es estratificar: elegir las instancias en cada una de las subclases, obligando a que la proporción sea la misma en el corpus de entrenamiento y en el de evaluación.\n", "\n", "\"Drawing\"\n", "\n" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Conjuntos de datos desbalanceados\n", "\n", "- Los conjuntos de datos donde el número de instancias de cada clase son muy diferentes, nos traen problemas para el aprendizaje, ya que un método que favorezca a la clase mayoritaria generalmente tendrá \"buenos resultados\" en términos de _accuracy_\n", "\n", "\"Drawing\"\n", "\n", "(Fuente de la imagen: User guide de la biblioteca imbalanced-learn)\n" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Conjuntos de datos desbalanceados: Oversampling\n", "\n", "- Oversampling: generar nuevas instancias de la clase minoritaria para equilibrar la cantidad de instancias de cada clase\n", "\n", "- Random oversampling: muestreo con repetición instancias de la clase minoritaria\n", "\n" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Conjuntos de datos desbalanceados: Oversampling\n", "\n", "- SMOTE: genero instancias \"sintéticas\" a partir de los vecinos más cercanos.\n", " - Dado $x_i$, considero sus k vecinos más cercanos y elijo uno ($x_{zi}$) y genero una nueva instancia $x_{new} = x_i + \\lambda (x_{zi} - x_i)$\n", " - Existen variantes de SMOTE que, por ejemplo, genera solamente a partir de instancias \"en peligro\" (al menos la mitad de los vecinos son de otra clase). \n", "\n", "\"Drawing\"" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Conjuntos de datos desbalanceados: Undersampling\n", "\n", "- Undersampling: reducir la cantidad de instancias de la clase mayoritaria\n", "\n", "- Random undersampling: elegir un subjconjunto aleatorio de instancias (Prototype Selection)\n", "\n", "- Nearmiss: elegir las instancias más cercanas a las de la clase minoritaria utilizando el algoritmo de vecinos más cercanos (Prototype Selection)\n", "\n", "\"Drawing\"\n", "\n", "\n", "- Mas info: imbalanced-learn User guide\n" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Conjuntos de datos desbalanceados: Undersampling\n", "\n", "- ClusterCentroids: dividir las instancias de la clase mayoritaria en N clusters y utilizar los centroides como nuevos instancias (Prototype generation)\n", "\n", "\"Drawing\"\n", "\n", "\n", "- Mas info: imbalanced-learn User guide" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Selección de atributos\n", "\n", "- Luego de que tenemos atributos \"candidatos\", nos gustaría quedarnos con aquellos que \"valen la pena\" para la tarea de clasificación que intentamos hacer. No hay una definición obvia de \"vale la pena\", pero nos interesan (para evitar ruido, y también por razones de eficiencia computacional) aquellos atributos que, en conjunto, sirvan para mejorar nuestra predicción. \n", "\n", "- El objetivo de la selección de atributos es eliminar atributos que son irrelevantes o redundantes. Por ejemplo: si tenemos dos atributos con valores idénticos, podemos eliminar uno de ellos. O si un atributo tiene siempre el mismo valor. O, por el contrario, todos sus valores son diferentes (en este caso, servirá como predictor perfecto de la clase objetivo si lo memorizamos, pero seguramente su capacidad de generalización será nula).\n" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Selección de atributos - Métodos de filtrado\n", "\n", "- Este tipo de métodos intentan evaluar (por separado) qué tan bueno es cada atributo. \n", "\n", "- Una forma muy básica: seleccionar los atributos que superan cierto valor de varianza. Se basa en la idea de que los atributos que valen siempre lo mismo seguramente no sean buenos predictores. Defecto: no utilizan la clase objetivo. \n", "\n", "- $ \\chi^2$ (chi-squared): cuando tenemos dos variables categóricas, podemos utilizar este método para estimar, a partir de los valores observadores y los esperados, si las dos variables son independientes. Cuanto mayor es el valor, mayor su correlación. Por lo tanto, podemos utilizar esta medida para obtener los atributos más correlacionados con la clase objetivo (y que, por lo tanto, deberían ser mejores predictores).\n", "\n", "\n", "- Ganancia de información: la ganancia de información, o información mutua de dos variables aleatorias es una medida (proveniente de la teoría de la información) que indica qué tanto podemos saber de una de ellas conociendo la otra, es decir qué tan dependientes son entre sí. Vale 0 sí y solo sí ambas variables son independientes. Por lo tanto, esta medida puede ser utilizada en forma análoga al test $ \\chi^2$ para eliminar atributos independientes de la clase objetivo." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Selección de atributos - Métodos _wrappers_\n", "\n", "- Este tipo de métodos de selección utilizan un método de aprendizaje para evaluar diferentes combinaciones de atributos y seleccionar la que obtenga mejores resultados sobre un dataset heldout separado, o vía cross-validation. Para que funcionen, el método debe ser capaz de asignar un valor de importancia a cada atributo, luego de entrenado el modelo. \n", "\n", "- Por ejemplo, el método de **eliminación recursiva de atributos** parte del conjunto inicial de atributos y, aplicando un método de aprendizaje (por ejemplo, árboles de decisión) sobre un conjunto de validación, elimina aquellos atributos menos importantes. Se repite el proceso hasta llegar a un cierto número de atributos deseados. " ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Selección de atributos - Métodos _embedded_\n", "\n", "- A diferencia de los anteriores, la selección de atributos es realizada por el mismo algoritmo de aprendizaje, durante el proceso de entrenamiento.\n", "\n", "- Un ejemplo ya visto en el curso es el de la selección de atributos que realizan los árboles de decisión.\n", "\n", "- Otro ejemplo, que veremos más adelante en el curso, son los métodos de regularización, que buscan generar modelos más \"sencillos\", para evitar el sobreajuste." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "Lectura: [An Introduction to Feature Selection](https://machinelearningmastery.com/an-introduction-to-feature-selection/) - Jason Brownlee" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "## Fase 3: Entrenamiento\n", "\n", "\"Drawing\"\n" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Entrenamiento\n", "\n", "- Durante el entrenamiento, utilizamos los datos del conjunto de entrenamiento, y un algoritmo de aprendizaje, para generar un clasificador.\n", "\n", "- Una vez generado el modelo, lo evaluamos en el conjunto de evaluación para obtener una medida de su performance (veremos más adelante las principales medidas utilizadas).\n", "\n", "- Los algoritmos de entrenamiento tienen usualmente hiperparámetros que deberían ajustarse (e.g. profundidad máxima de un árbol en los árboles decisión). Usualmente lo que se hace es probar diferentes valores para cada parámetro y ver cuál obtiene mejores resultados. Como no queremos hacer esto sobre el corpus de evaluación (¿por qué?), utilizamos un subconjunto del corpus de entrenamiento, el corpus de validación (o corpus held-out). " ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Titanic: entrenamiento\n" ] }, { "cell_type": "code", "execution_count": 49, "metadata": { "slideshow": { "slide_type": "slide" } }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAABT4AAAHxCAIAAADLPTSEAAAABmJLR0QA/wD/AP+gvaeTAAAgAElEQVR4nOzdf3xbdb348c+BwRjoDYpuDrRTGIxRsR1zsx0/xjoQx4MTf7VbWyz48K4lFbkgq7grqV+8zdVNU5WL3JWkIlJtsnaKy9E7YE1wwGiZcyTIhHbblVRRG0USFa7Ixvn+8XHHQ341bdKetHk9/9gjPZ9PznmftJ+z887nfD4fRdd1AQAAAAAAitUJVgcAAAAAAACyIXUHAAAAAKCokboDAAAAAFDU5lgdAAAAeRkcHPz1r39tdRSYbqtWrXrnO99pdRQAAEwThWnqAAAzWl1d3Y4dO6yOAtNt+/bt69evtzoKAACmCb3uAIAZr7a2tr+/3+ooMH0URbE6BAAAphVj3QEAAAAAKGqk7gAAAAAAFDVSdwAAAAAAihqpOwAAAAAARY3UHQAAAACAokbqDgBAKUokEl6v1263K4pit9v9fn8ikchUWUlnOqMFAKDEsTgcAAClaPPmzV1dXfK1pmmapqmqGggEUmuOjo5Ob2gAACAZve4AAJScSCTS1dXldDqj0aiu69Fo1OFwaJo2MjKS6S1ut1t/o+kMGACAEkfqDgDAjBSJRDo7Oyf33n379gkhmpqaysrKhBBlZWUOh0MIceDAgdTKhw8fFkIsW7Zs8rECAID8kLoDAJCvUCjU2toqR4C3t7dHIpG0pXa7PRQKyY1er1dRFHPNkZERRVG8Xu+4hxsaGmptba2srGxra5Nb0o5FzzIiXT4Dv2DBAmPLwoULhRAHDx6c2JkDAIBpQeoOAEBeNE1bu3atMW7c5XJVVlYaKXp7e7tRKmu2t7cLIZqbmx0OR3t7u5wcLpFItLW1qara3Nyc6UCJRELTNLvdXl1dLYQIBAJjY2OTi9nlcgkhbDabsWX+/PnG9iRPPfWUEOKMM86QXzfI7xeyzGkHAAAKjtQdAIC82O12IYQcNK7r+uDgoBCiv79fCBEKhVwul9PpjMfjuq7H43Gn0+lyuWRn+7//+79rmtbd3S2E6O7u1jTtW9/6VtpDjI6O+v3+008/3ev1NjY2RqPRbdu2qaoq820hhJ5ZoU6zsrKypaVFvm5paWlqaiJ7BwBg2jDDPAAAeVFVVdO0/v7+ZcuWLV++vKqqykiYH3nkESFEW1ub7N+22WxtbW0ul2tgYKCioqKsrCwQCMjMv62tLRAIyJHnqRYtWiSE8Pl89fX103RWJvKx/MHBwaqqKrnF7/c3NDTs2rXLkngAAChBCjPEAgBmtLq6OnG8l9sSkUiksrJSvlZV9ZZbbqmpqZE/Zln83Pj/t729XfbMd3R0ZKo8Ojr6xBNPNDQ0qKra2Ni4atWqpCQ/lwOl1k8qSrsxE0VRMi0mNw0URdm+ffv69estOToAANOPB+YBAMhLRUWFruvhcNjtdsvR7Ha7PWmmujyVlZXV19fH4/Hm5ube3t5Fixa1trZqmhaLxSa3Q6fTKYQwP/EuX8vtOdI0bXJHBwAAE0XqDgBAAVRUVGzatCkajQaDQU3TZD+8XHFNDnRPOwRd0zTZ5e5yucbNhG02m+zolsPp7Xa7MUX8RMe6l5eXCyHMs9w9//zzQoi0T+zb7XZFUVLzfHl2AABgGpC6AwCQF7nw29DQkBCirKxs8eLFRpF8mN/tdhvd46FQSFEUuR776Oio3W53u90dHR1ut9tut8s128ZVVVW1bds22c8/uZiXLl0qhOjp6ZFHHB0d3bFjhxBi5cqVqZUbGxuFELt27TK2yNfy7AAAwDRgrDsAYGazfKz70NCQXK3NzOPxyGXe5FB2c5Gqqt3d3fPnz29tbX3hhRd6enpsNlsikWhqajrrrLO2bds2PWHb7fakfn6Hw2Ec3TzuXcaWVDn74Pypxlh3AECpYYZ5AADyUlVVFQ6Hd+zYIVN0p9O5cuVKVVVlaUdHR3l5+Z49e+TS7h6P58Mf/vD8+fO9Xm9XV1c4HDYmn+/o6KisrLzooouyLO1eQN3d3Tt37tQ0TdM0VVVVVc2UCdtstp6enl27dvX29mqa5nA46urqjKn4AADANKDXHQAws1ne647pR687AKDUMNYdAAAAAICiRuoOAAAAAEBRI3UHAAAAAKCokboDAAAAAFDUSN0BAAAAAChqpO4AAJSiRCLh9XrtdruiKHa73e/3JxKJLPX9fr+s3NraGolE0tbRNE0uCJ90IOO9drvd6/XGYrGCnQYAAKWBxeEAADObtYvDdXZ2trW1zcT/TFtbW+VS8wZVVQOBQNrKdrtd0zTzFp/PV19fb94SiUQqKyuFEOZPI5FINDU1Jb1XVdXu7u758+dPOngWhwMAlBp63QEAmLy2tjarQ5iMSCTS1dXldDqj0aiu69Fo1OFwaJo2MjKSWtnv92ua5na74/G4ruu6rvt8voaGhtHRUaPO0NCQzNuT7Nq1S9M0j8cj3xuPx51Op6ZpPT09U3h6AADMOqTuAACUnH379gkhmpqaysrKhBBlZWUOh0MIceDAgdTKvb29QoiNGzfabDa5Zd26dUKIhx56SP7Y2dlZXV3t8/kyvbe5uVm+12azyS87ZuhXHgAAWIXUHQAAIYQIhUKtra1yPHYoFDK2K4qiKEosFuvs7DSGhRtF5jrGi9HRUbvd3t7ebuzEPNjbeLv5jX6/X742xpzb7fbW1lZzhIlEQg41Tw1eySztycoO8wULFhhbFi5cKIQ4ePBgamX5uLuRtxuvjTy/ra0tEAgkPT8vBQKBpNEE5v0AAIBc6QAAzGS1tbW1tbV57sTpdCb9/+h0OmWR/FFVVXOpz+fTU8a3G1vk3mSdXHaeNMJcVVVd14PBoBAiHA4bQQ4ODorjyXCSif5Hn7YoU3157sbT8rqux+PxCe3BbHh42PzhTI4QYvv27fnsAQCAmYVedwBAqQuFQi6Xy+l0msdju1wu8zzqFRUVslRm1PI5cP2NyapRuby8XNd12Qtt7NwYVS53bu7Y93q95lJN00KhUE1NjaqqO3bsMKrt3btXCLFkyZLUU8jyP33+n09jY6MQYteuXfLHRCLhdrsnvbeenh5VVeUj9wAAIEek7gCAUvfII48IIdra2pLGYw8MDBh1brrpJllaU1Mjjj9DnomsI8mp75ubm41R5c3NzeKNU+K73e60pbfccovL5TJmg2tra1NV9bzzzivIWedu3bp1qqo2NDTIJ/BPP/30Se+qvb3d5XJ1dHTw2DwAABPC4nAAgJkt/8XhMg0IF0Loui5Lzf9dmrcklWavnLoxe6kQwm63r169etOmTXLpNY/HI3P7CZ1CpvpZjpskFovt3LmzpaVFVdXGxsb6+vqJ7kEcz9vD4XBFRUWmaHPE4nAAgFJDrzsAAMWrubm5ra0tFos9++yzQoiVK1cWZLdy+L2cD0+Sr1OH5Uvz589vbm7WdV1ORycfBMj9sflYLNbe3h6JRIaHh/PP2wEAKEGk7gCAUifXRTNPw1bAgeJy5+Yl0OXa6XK7lFpqpNCXXXaZECIUCsnR9Zny3omOdS8vLxdCjI2NGVuef/55IYR8bj+JnBvfnOcfPnxYCHHWWWdlO/PjIpHIxo0bhRDd3d3T/7Q/AACzA6k7AKDUyUfu3W53LBaTW0KhkKIonZ2dOe7BnNam3bnX65X5+ejoaE9PjxDi6quvNuqklq5Zs0YW2Ww2n8/X0NCgaVqmLvFJWLp0qRCip6fHOK6cDy9tr76cpq6vr0/+ODIyIocnrFq1atwDjY6OVlZWVlRUdHR0zJ8/v1DxAwBQcnKbiB4AgCI1RYvDqao6Njamp1vwzLzFWDTO4XCkrZx250mLwyVVMEolYyW2wcHBPE/TLGm5O+MUUs8xHo+nVk67ulvq6Xs8nqm4AxEsDgcAKDH0ugMAIDo6Onw+n/EQu8fj6e7uzqWXuKOjQ77rhRdeyL5zmf2qqurz+To6OpIqyHHjqqoGg8GkUmMy9rPPPntiZ5VVd3e3x+MxovJ4PFu2bElb02azycryR6fTGQ6H5dJ342ppaSlUwAAAlDJmmAcAzGz5zzBvoexTsksjIyNLlixxOBzbtm2brriKHTPMAwBKDb3uAAAUNTn63TytHQAAKDVzrA4AAACkZ6zW7nQ6WVMNAIBSRq87AABFSg5Ed7vdSaPfAQBAqaHXHQAAy2Qf5R4IBKYtEgAAUMzodQcAAAAAoKiRugMAUFwURTFGuVvL7/fb7XZFUVpbWyORiLkokUgYpXa73e/3JxKJtDuJRCJFcjoAAMxcpO4AACANu93e0NCgaZoQoqurq7Ky0u/3y6JYLNbU1GSUaprW0NDQ1NQUi8WSdhKLxSorK6c5cgAAZh9SdwAAkMzv92ua5na74/G4ruu6rvt8voaGhtHRUSHEzp07NU3z+Xz6cT6fT9O0nTt3Ju3n//2//2dF+AAAzDak7gAAIFlvb68QYuPGjTabTW5Zt26dEOKhhx4SQrS0tAgh6uvrjfrytdxu6OzsfOGFF6YrZAAAZjNSdwAA/ikUCrW2tsrR5u3t7eYB3pFIpLOzUxbJ0d1GkTE6XdM0WSqfJBdC+P1+WZq2vrk001hxc1R2uz0UCuUYsJmSWdr6Mn4jbzdeHzhwQBxftS6VeXsoFGpra2NZOwAACkMHAGAmq62tra2tLciu0i7GFgwGMxUZT4zLH5PqhMNhp9OZe31VVc2lRlRJOxFCOJ3OcQNOMtE7AZmEG0/L67oej8eN+vK4SQ/MyzOSPw4PDxsVshxl0oQQ27dvL+w+AQAoZvS6AwDwD3a7XQgRjUbl/5GDg4NCiP7+fqNocHBQFkWjUSFEQ0OD+e379u2TuW4wGBRCyOnZzFuS6nu9XnmsaDTqdDo1TUvqURdChEIhl8vldDrlfuLxuNPpdLlcsnc9S8BJstwKpP0oGhsbhRC7du2SPyYSCbfbbZSqqhoMBnt7e42u+97e3mAwKBP+RCLR1tbmdDrNT9QDAIB8kLoDAPAPMvPs7+8PhUKJRKKqqkrX9W3btonjqe/ZZ58diUQ0TfN6valvv+mmm+RT5TU1NXJLW1tb0hYzt9tdVlYmhCgrK2tubhbpsu5HHnnEvB+bzdbW1iaEGBgYyB5wntatW6eqakNDg8zMTz/99KQKTz31lDEoQAihadqRI0eM89I07aabbso/DAAAICmZvm4HAGBGqKurExm6micqEokYK5mpqnrLLbeYU+729naXy5X0FvnfqBwxbv4vNfuW1FLzxtSaaem6nj3g1J1n2k/a7bFYbOfOnS0tLaqqNjY21tfXG4H5/f6Ghgafz2f0qxtbhBANDQ2Dg4NVVVWZPor8KYqyffv29evXF3CfAAAUM1J3AMDMVsDUXYpEIgMDA7JzW1XVjo6OiooKr9fb0tLicDjq6urOOOOMhQsXLliwQFidumcJOKnyJFL3JKOjo4sWLXK73Zs2bcoUv6qq5q74SR9rXKTuAIBSwwPzAAC8QUVFxaZNm6LRaDAY1DRNdmvLZc+2bdtWU1NTUVExd+7c/A8k10iXRkZGhBCpM9I5HA7xxuniUseopw04yUTHutvtdkVRzJPeHz58WAhx1llnZTmj7Hk7AACYNFJ3AAD+Qa6yNjQ0JIQoKytbvHhxUgWZYCfN2TZpXq9XZu+jo6M9PT1CiDVr1iTVkc8UuN3uWCwmt4RCIUVROjs7cwl40uQ0dX19ffLHkZER+VzDqlWrZDwyEiO3l0vfud3uTF8NZPmaAAAAjGuO1QEAAFAsrr/++q6ururqavNGj8cjhPD5fA0NDUuWLEl6y8jIyHnnnTfpIy5atMh47XQ6U0eq19TUyCnlzcPsVVVtamrKHnCe5DR1LS0t8nEDyefzyXn1mpqa9uzZs3btWvNbjKgAAEDB0esOAMA/VFVVmRdjdzqdgUBAzv1eX19vpMROp3N4eDgcDgsh9uzZM+nDdXR0yO5rudZaR0dHpmo+n08+OS+E8Hg83d3d8+fPzx5wnmw2W3d3t/mUw+GwMSnd/Pnze3p6fD6fnOJeVVWfz9fT0yOjAgAABcc0dQCAma3g09RNg6mYdL2kME0dAKDU0OsOAAAAAEBRI3UHAAAAAKCokboDAAAAAFDUmGEeAIDpxih3AAAwIfS6AwAAAABQ1EjdAQDIRlEUOSF8MYSRFMzIyEh7e7vc6PV6Y7GY+S1ZShOJhN/vt9vtiqLY7fbU92aRSCS8Xq/cbXt7+8jISFJplj1nKk17dgAAwMDicACAmW2qF4crkoXczDmtDCYSiVRWVprrqKra09Njs9mylyYSiaamJk3TkkqN5eKzs9vtSe8Nh8MVFRVCiOx7zlK6YMGCpLPLjsXhAAClhl53AABmDF3XZWabSCQqKytVVY1Go7qux+Nxt9utadquXbvGLd21a5emaR6PJx6Py1Kn06lpWk9Pz7gB+P1++V4ZSTAYFEJ0dXXJ0ux7zlJqnBcAAEiLXncAwMxWUr3uRhhDQ0PV1dU+n6++vl5uSSQSp59+uqqqgUAge6nsNjefkSwVOZymfG88Hpd9+0mBZd/zuMfN/aOm1x0AUGrodQcAzHKJREJRlNbW1qTtra2tiqIkEgkhRCQS6ezslGOt7Xa73+9Pu6vUwdipW0KhkNyz3W4PhUKZolIyy+Wk9u7dK4RYtWqVscVms+m6HggExi0NBAJJ6bGRh49LvteoL59+9/l85tJMe87nuAAAlDhSdwDALGez2dxud1dXl3m+tFgs1tXV5Xa7bTabpmmVlZVtbW2ySNO0hoaGTNl7du3t7WvXrpUPkGuatnbt2vb29oKcRZI9e/YIIcrKyoxZ3zo7O40TzF6aSk41Z2TguZDfdNjtdnPf/kT3PInjAgBQmkjdAQCz3xVXXCGEMPeBy9eqqgoh7Ha7EGJwcFCOuI5Go0KIhoaGiR4lFAq5XC6n02key+1yuSKRSGplPbNcjiW7u9vb2xsaGuTrtra2jRs3yocIspem6unpUVV13bp1uZ/ssmXL3G63qqrZv+bIvudJHBcAgNJE6g4AmP0qKipUVe3t7TW29Pb2OhyO8847TxzPos8+++xIJKJpmtfrndxRHnnkESFEW1ubfA7cZrPJnvyBgYECnEMGY2NjMn6fz2dMRJdjqdTe3u5yuTo6Oib0+HpNTc2mTZsCgYDH42loaEg7NCD7nid3XAAAShPT1AEAZrYcp6kLhUJr164dHh4+77zzRkZGlixZEgwGa2pqZKlMI5Pekjp3Wuo8aqmlaaX+bzuhyqmHlj+ap4szT0SXvdS8W3nixupukzC5PactZZo6AAAyodcdAFASli9fLo4PAj9w4ICxRQjh9XpdLpfD4QgGg+FweGxszMI4c+R0OsUbp3mTr+Xj8dlLpVgs1t7eHolEhoeHJ523T2LPhTouAAAlhdQdAFASbDabx+NpaWmJxWINDQ0ej8fIbFtaWoQQ27Ztq6mpqaiomDt3bo77TJr4zeFwCCHkQPdxh6/nOda9vLxcCDE6OmpskePYZQzZS4UQkUhk48aNQoju7m45aiBHctI785h5+SHkuOdJHxcAgBJH6g4AKBWrV68WQixYsEAIcdVVVyWVytnOE4mE2+3OtAc5rd3Q0JCsedddd5lL5aP7brfbSOlDoZCc3b2AZyHJhd+8Xq+RRctx7FdfffW4paOjo5WVlRUVFR0dHfPnz5/QcRsbG4UQfX198sdEItHT0yOOn3v2PedzXAAASl2Wb/0BACh+tbW1tbW1OVaWncMOh8O8MdPiZMPDw/rxPvC0NY0k39iVfFLdTFVVY664fKT+r50atvm8spR6PJ4sdwXj3h7I7y/MnE5nLnvOXprLoc2fxvbt23P97AAAmPnodQcAlBDZOXz99debN9bX1xtZpdPpHB4eDofD4vjA+KSaPp9P5q4ej2fTpk1JFTo6Onw+n/H0uMfj6e7unqIe5vr6+sHBQXksVVV9Pt+2bdtyKZUDBCYtEAgYH4KcIKCjoyOXPed5XAAAShkzzAMAZrYcZ5if6XKffb0gx7Lk9oAZ5gEAyIRedwAA8E9DQ0NZnmwHAACWIHUHAGDGUBQly4LwBbF3797m5uYpPUSqaTgvAABmNFJ3AADwT6kD+AEAgOXmWB0AAAAY3+yem2Z2nx0AAPmj1x0AAAAAgKJG6g4AAAAAQFEjdQcAAAAAoKiRugMAAAAAUNSYpg4AMOMNDQ3V1dUVfLcvv/zyaaedVvDdlppXXnll3rx5rP0GAEA+SN0BADNbdXX1VOz2V7/61VNPPXXJJZfMnz9/KvZfInRdf+yxx+bNm1dVVXXyyScXare1tbXvete7CrU3AACKn8JyLAAAmB07duz222//6le/etttt335y18+4QQGl+UlHA5/9KMfPXbs2AMPPLB8+XKrwwEAYEbidgQAgH/6y1/+8pGPfOTOO++8//77t2zZQt6ev8rKyv3795933nmXXHLJ/fffb3U4AADMSNyRAADwD0eOHKmqqjpw4MCjjz76iU98wupwZo8zzjjjwQcfvPnmm6+//vqbb7752LFjVkcEAMAMwwPzAAAIIcRjjz328Y9//J3vfOfOnTsZRz1Fent7N27ceNlll/l8vre85S1WhwMAwIxBrzsAoACUrKyObnwej2ft2rVr1qx5/PHHydunTmNj4969e5977rkVK1Y888wzhd15Pn+BM+UPFQBQskjdAQAl7ejRozfffLPD4bj11lv9fv+pp55qdUSz3LJly/bv319WVlZdXb1jxw6rwwEAYGYgdQcAFIBukmlLEfrTn/70oQ996Nvf/vaOHTu2bNlCv+v0eNvb3vbwww/feOON69ev37x58+uvv17AnevpFHD/AABYgrHuAIACkwlw8f//cujQIbvd/pe//GXnzp0sWmaJ733vey0tLZdffnlvb+/pp5+e597y+cObKX+0AICSRa87AGA6yLHEo6Ojdru9vb1dpBtdnLolFAq1trYqimK320OhUAHjefjhh1euXPmWt7xl//795O1W+cQnPvH4448fPHhwxYoVBw8enOrDRSKRzs5O+Wdmt9v9fn+mmsYfnqIo7e3tkUgkbWnB/ywBAMgo7XNlAABMWtr/X+RGp9MphPD5fGmrJW2Rlc2cTmdBIrznnnvmzJnT0NDwyiuvFGSHyEcsFrv88svf9KY3/eAHP8hnP9lvbAKBQOpdkPxTTHpv2prBYFCWTt2fJQAAWdDrDgCYPuXl5bqu19fXj1szFAq5XC6n0xmPx3Vdj8fjTqfT5XIl9X9O1NGjRz/zmc84HI7bb7+9t7d33rx5+ewNBfH2t7/9oYceamhoqK2tzX/oe6bp5e12uxBicHBQ3gBFo1EhRENDQ+oeZM1oNCprDg4OCiH6+/vFlP1ZAgAwLlJ3AMD0qampybHmI488IoRoa2uz2WxCCJvN1tbWJoQYGBiY9NFffPHFK6+88rvf/e4DDzxwxx13THo/KLiTTz7Z4/F0dXV94xvfUFU1Ho8X/BAyDz/77LMjkYimaV6vN1NNVVWFEP39/aFQKJFIVFVV6bq+bds2MTV/lgAA5IJp6gAABZZ2xq/Ujdm3ZJnsfXL/cz3zzDN2u11RlEAgUF5ePok9YBo88cQTtbW1b37zmx944IELLrhgQu8dd6q59vZ2l8uVtNH89yZfRyKRyspKWaqq6i233GJ85VTwP0sAAHJErzsAYPbbtWvXJZdccuaZZw4ODpK3F7NVq1bt37//rW99a1VV1Y9+9KMC7tnr9bpcLofDEQwGw+Hw2NhYppoVFRW6rofDYbfbrWna2rVr7XY7j8QDAKxFrzsAoMAm1+sei8UWLFhgbGltbe3q6orH4/LJ5Hzceeedt95666c+9am777775JNPznNvmAavvvrqjTfeeO+99952221f/vKXTzghp56G7L3uSaWJREIuR5fa6242Ojp6+PDhtWvXytIC/lkCADAh9LoDAKwhRxQPDQ0JIRKJxF133WUuraurE0K43e5YLCa3hEIhRVE6OztzP8Srr776yU9+ctOmTV/+8pe9Xi95+0wxd+7c7u5uOfT9wx/+cCKRKNSeR0ZGhBCJRMLtdmeqIxd+k3+ZZWVlixcvNooK8mcJAMAk0OsOACiwHHvd/X6/eX5vt9stZ/wy6qSOTFZVtbu7e/78+bmE8cc//vHjH//4U0891dvbe80110zyZGCpxx9/vK6uzmaz/ehHPzr//POzV87e657092YYHh4+77zzzO8dGhqqrq5OqubxeJqbm0Xef5YAAEwOve4AAGvU19f7fD7Z9+7xeDZt2pRUoaOjw+fzORwO+aPH48k9QXr66aff//73//a3v33yySfJ22euSy65ZP/+/Tab7QMf+MDOnTvz2VV9fb3H45GvnU7n8PBwOBwWQuzZsyepZlVVVTgcNtZvdzqdgUBA5u0ivz9LAAAmjV53AMBs84Mf/OD6669fvnz5jh073v72t1sdDvL16quvtra23nfffbfddttXvvKVLNO8AwAwW9HrDgCYPXRd37p16/r166+99tqBgQHy9tlh7ty59957b1dX19e//vX6+vqXX37Z6ogAAJhu9LoDAGaJv/3tb83NzX6/v7Oz89/+7d+sDgeF99hjj9XV1S1YsOCBBx44++yzrQ4HAIDpQ687AGA2+O1vf3vZZZf9z//8z4MPPkjePltdeuml+/fvnzt37ooVKx5++GGrwwEAYPqQugMAZrxwOFxVVRWPx5944gm5BDdmq3e+852PPvqoqqpXX3311q1beXgQAFAiSN0BADNbX1/fxRdffMEFF+zbt2/JkiVWh4Mpd8opp9x3333//d//3d7e3tjY+Morr1gdEQAAU47UHQAwU8lJ6err6z/xiU/8+Mc/Pv30062OCNOnpaVl9+7doVBo1apVv/rVr6wOBwCAqUXqDgCYkV5++eXa2tovfvGL3d3d99xzz5w5c6yOCNNt9erV+/fvnzNnzooVKwYGBqwOBwCAKUTqDgCYeV544YXVq1fv2bPnoYce+tSnPmV1OLDMu971rscff6i2jVQAACAASURBVPzqq6/+0Ic+tHXrVqvDAQBgqtBHAQCYYQYHBz/2sY+9/e1v/9nPfvae97zH6nBgsVNOOeW73/1ueXn57bffHolEuru7Tz31VKuDAgCgwOh1BwDMJD6fb+3atcuWLXvsscfI2yEpivL5z3/+Jz/5yYMPPnjxxRc///zzVkcEAECBkboDAGYGXdfvuOOOxsbG5ubmH//4xzabzeqIUFyuuuqqn/3sZ0ePHl2xYkUoFLI6HAAAConUHQAwA/z1r3/96Ec/umXLlvvuu+/OO+884QT+/0Ia55xzzuDg4OWXX37VVVcx9B0AMJsouq5bHQMAANn87//+74c//OHf//73P/jBDy677DKrw0Gx03X9q1/96he+8IWGhgav1ztv3jyrIwIAIF+k7gCAorZ3796PfexjCxcu3Llz56JFi6wOBzPGrl27Ghsbzz777B/+8If85QAAZjoeOAQAWC8Wiw0NDaVu7+7urqmpufTSS/fu3Uv2hQlZt27dz372s1dffXXFihU//elPUys8//zzf/zjH6c9LgAAJoPUHQBgvc997nPXXHPN6OioseXYsWObN29uaWn57Gc/29fXd9ppp1kYHmaoxYsXDw4OXnrppVdeeWXS0PdXXnlFVdVbb73VqtgAAJgQHpgHAFjsiSeeuOSSSxRFWbp06ZNPPnnaaaf95S9/aWxsHBgY8Hq9n/jEJ6wOEDObMfT92muvveeee+bNm6frekNDQ39/v67roVDo8ssvtzpGAADGQeoOALDSsWPHKioqhoeHjx49OmfOnKuvvvprX/vaRz7ykUQi8aMf/WjFihVWB4hZ4ic/+cm11157wQUX/OAHP+jp6dm8ebOu6yeeeGJZWdkvf/nLU045xeoAAQDIhtQdAGCl//qv//rsZz/7+uuvyx9POOGEM88888wzz3zggQfOPPNMa2PDLPPss89+5CMfefHFF1966SXjT+7EE0/84he/+MUvftHa2AAAyI7UHQBgmbGxscWLF//1r39N2v6d73znk5/8pBURYZb7xS9+sXLlyr///e9G6i6EOOmkk55++unzzz/fwsAAAMiOaeoAAJb53Oc+9+qrr6Zub2lpefLJJ6c/Hsxuf/3rXzds2HDs2DFz3i5t3LiRzgwAQDEjdQcAWGPv3r3f+973XnvttdQiXdevueaa3/zmN9MfFWYrXdc/+clPHjp0KPVP7rXXXnviiSe+973vWRIYAAC54IF5AIAFjh49+r73ve/QoUNHjx7NVGf58uWPPfbYvHnzpjMwzFZf+cpXvvCFL2QqVRTFZrMdPnz4jDPOmM6oAADIEb3uAAAL3H333XJW+aTtJ5xwwgknnDBv3rzrrrvO7XbPnTvXkvAw+8yfP//iiy9WFOWkk05SFCWpVNf1l19+ua2tzZLYAAAYF73uAIDplnZ2ujlz5hw7dmzlypWf+tSnGhsb3/SmN1kVHmaxF154YceOHd///vd/9rOfnXzyyX//+9/NpYqiBIPBNWvWWBUeAACZkLoDAKbbtdde29/fL4ccn3TSSa+99tp73vOe66677rrrrjv77LOtjg4l4Ze//GVfX993vvOd0dFRI4c/8cQTFy1a9Mtf/pLHPQAAxYbUHQAwrR599NHLL79cvj7ttNOuvfba66+/vrq62tKgUKJ0XR8cHOzt7e3t7X3ppZdOPPHEY8eOdXR0OJ1Oq0MDAOANSN0BzFqDg4Nf//rXrY4Cb6Dr+sDAwJ///OcFCxa8+93vXrhw4Yknnlionff39xdqV5gRCtjGdV0fGxsbHR194YUXdF3/4Ac/yJCNolVdXX3rrbdaHQUATLc5VgcAAFPl17/+9Y4dO2pra60OBP8Ui8UWLVpUVlZ2yimnFHC3v/nNb4aGhgq4Q8wIBWzjiqK84x3veMc73nH06NHf/va3v/vd784999z8d4uCo6UDKFmk7gBmOXpiS0FfX9+GDRusjgLWoI2XlLq6OqtDAABrsDgcAAAAAABFjdQdAAAAAICiRuoOAAAAAEBRI3UHAAAAAKCokboDwBskEgmv16soiqIo7e3tIyMjE92DfG8+MeS/B8vJj9FutyuKYrfb/X5/IpHIVFlJZzqjBUqHpmmp7WtC1z0aLABYgtQdAN6gqamppaVFvna5XEuWLIlEItaGNBNt3ry5paVF0zQhhKZpDQ0NTU1NaWuOjo5Ob2hA6YpEIna7PXV77tc9GiwAWIXUHQD+ye/3a5rm8Xh0Xdd1PRgMCiG6urqsjmuGiUQiXV1dTqczGo3quh6NRh0Oh6ZpWbry3G63/kbTGTBQCoaGhiorK1O3T+K6R4MFgOlH6g4A/9Tb2yuEWL9+vfyxpqZGlGrqHolEOjs7J/feffv2CSGamprKysqEEGVlZQ6HQwhx4MCB1MqHDx8WQixbtmzysQIlIJ8mKYTo7Oysrq72+XypRRO67tFgAcAqpO4A8E+BQEDXdZvNJn+Uz3sbN7tySOfo6Kjdbm9vbzfe5ff75aBuv98/ocMlEgnjva2trZk6peUtuzy6HDduLg2FQq2trcYgVfNjrlmKshgaGmptba2srGxrazOfeO4DXOUjtQsWLDC2LFy4UAhx8ODBXAIAilb2xihMVwM5YjypmRhN0m63h0KhHA+af5MUQrS1tQUCgfr6+tSi7Nc9AECx0AFgltq+ffukr3Jut1teJH0+n7FRbnE6nebtsj/ZYLwxl6Ooqpp0TQ6Hw8aBZJ1AIJB66TaOnrY0GAxmL0orHo8HAgEZksPhCAQCY2Nj5hPP/T+RtEWZ6stPLBwOezweWcfj8cTj8Vw+QEM+v2vMXNP8e8/eGHVdlxeHTM0ktdTpdGY5XAGbpFmWammve2nr5Nlg81FbW1tbWztthwOA4sGNDoBZK5/b+mAw6Ha75U2zcRebelMrB4WaB3Ubd+fjHkKmAU6nU971yj4uh8Ohv/HeWr4eHByUP0aj0dRSeXRd1wcHB5N2krYoSTQalUdXVdXn8xlvmbS0n0Cmj8XIFsxUVZ1QMkDqXpqm+feevTGmXg2M7/XMpfIPOx6Py2uF/LYuScGbZOpZpC1Ke91LUpAGmw9SdwAlixsdALNWQW7rZc+S7KyWN6lGx5d+vBvNfGM9PDycY+oub+vNezOk7mFsbCwcDgcCgaSvBuRNttvtDgaDSbfOWYrSHi5LP9tETSh1T0qH9OPfYkwoHlL30mTJ7z1TY8x+NZCl5pYYj8dFuvne9Clokqk7z17HfN1L+/Y8G2w+SN0BlCxudADMWgW5rZe316qq6ulueSeUo+ZeLako7VO4sigcDps7vsy32lmKkozbxZd69KQwcjm1HD8Wo7L8zHNE6l6apv/3nqUxZv+zn1AjKniTTBtSFubrXi4m2mDzQeoOoGQxTR0AZCOnbpLzNlnC6/W6XC6HwxEMBsPh8NjYmLm0oqJC1/VwOOx2uzVNW7t2rd1ul9PRZSlKUlZWVl9fH4/Hm5ube3t7Fy1a1NraqmlaLBabXMwyvUkkEsYW+TrTSOC0LPzMgbSyN8YCKniTnKhJXPdosAAw1UjdAeCf5NTQ5pxT3isnzUVnkLmoeWZ4Obl6LuQ+x70Xb2lpEUJs27atpqamoqJi7ty5qXUqKio2bdoUjUaDwaCmaealm7MUJbHZbKqqBgIBOSrebrcbU8Rn+QI47a7Ky8uFEObE5vnnnxdCyLXikqR+5vJ1ps8csEr2xiivBuYrgPm1/HtOO3Ql0+EK2CSzm9B1jwYLAFYhdQeAf2psbBRC9PX1yR8TiURPT48Qoq6uLm39NWvWCCHa2trkPfro6KjX683xWKtXrxZC3HXXXfLG1+/3yyXi0laW3w4kEomkOaLkQlNDQ0NCiLKyssWLF+dSlF1VVdW2bdtkd32Ob0mydOlSIURPT4/xsezYsUMIsXLlytTK8jPftWuXsUW+zvSZA9bK1Bjl1cDr9aa9Gsi/Z7fbbXxbFwqFFEXJZan2/JtkdhO67tFgAcAy+T1vDwDFa3LjYFMXbDMWcEp72Ux6CNxYMGlyx0pdHC7T6srDw8P68Xnjk3g8nuxF0yD11MyT25tPMB6PZ/nMc8RY99I0zb/37I1Rn/jicKqqpp2ockqZQzJkb4MFb7D5YKw7gJLFjQ6AWWvSt/U+n89YS9k8u1vaW15zfTnHcqZqqeLxuJHqO51OIwFI2kNSHTn/nJGEh8NhIyVwOp2BQMB4Y5aiqTY2NubxeOTHoqpq0srPSScYj8czfeY5InUvTdP/e8/eGHXT1UBWSPpT9/l8xoPlHo9n+vN2PYfrWGobLHiDzQepO4CSpeiTGhYFAMWvr69vw4YNXOVKAb/r0lT8v3dFURwOx7Zt26wOZPaQT+b39/dbHQgATDfGugMAAORLURRjdgkhRCKRkOPY5awWAADkaY7VAQDA7KQoSvYKxdxVCGCiAoGA3W6vrq42b1RVdd26dVaFBACYTeh1BwAAyJeqqsFg0JhdwuFw+Hy+np4euUY6AAB5otcdAKYEnepAqampqampqeno6LA6EADALESvOwAAAAAARY3UHQDwT4lEwuv12u12RVHsdrvf708kElnq+/1+Wbm1tTUSiaSto2la6sj/RCJhvDeXAwEolIk285GRkfb2djkPn9frjcViqXXSNnOR2yUCAJATK1emA4CpZOFa3263e4ZeYI1Fpw2qqmaqLBd2NpMr25vJha+TPo2xsbHU96qqOullrlnXvTRZ+3svkWZuNGFz5Xg8nrZO0ntzuURMFOu6AyhZ9LoDQOG1tbVZHcJkRCKRrq4up9MZjUZ1XY9Gow6HQ9O0kZGR1Mp+v1/TNLfbbdzE+3y+hoaG0dFRo87Q0FBlZWXqe3fu3Klpmvkm3ufzaZq2c+fOKTw9oKBKoZknEonKykpVVWXleDzudrs1Tdu1a5dRJ1Mzz+USAQDIHak7AOAf9u3bJ4RoamoqKysTQpSVlcneuQMHDqRW7u3tFUJs3LjRmEBbLoL10EMPyR87Ozurq6t9Pl/qe1taWoQQ9fX1xhb5Wm4HMHUm1MyfffZZIURjY6OsbLPZNm7cKI43f5G1mY97iQAATAipOwBkEwqFWltb5YjQUChkbJfDPmOxWGdnpzFe1Cgy1zFejI6O2u329vZ2YydJg72Tdi4ryNfGYFS73d7a2mqOMJFIyHGkqcErmaU9WdkbtmDBAmPLwoULhRAHDx5MraxpmhDCvPCVfG0kAG1tbYFAwJyfG1Ifo82+HZhSNHORoZnv3btXCLFq1Spji81m03U9EAjIH7M083EvEQCAiZnmB/QBYNrkPw7WWKLZ4HQ6ZZH8MSnVlE+Ap15mjfcK01DPcXdu3Bwbx9J1PRgMCiHC4bAR5ODgoKycGv9EL/5pizLVl+duHvIaj8dz3IM8taQH5jOdRS4Y616aCvJ7p5ln2qgfb+a6rvt8Pvna7XannZMidQ+5XyImhLHuAEoWve4AkF4oFHK5XE6nU956xuNxp9PpcrnMkyRXVFTIUnmrLR8Q1d94F2tULi8v13Vddk8ZOzeGm8qdm3v8vF6vuVTTtFAoVFNTo6rqjh07jGqyW2zJkiWpp5Dl6p//59PY2CiEMIa8JhIJOWtXLlRVDQaDvb29Rvdgb29vMBik1x3TjGaenew5b29vb2hokK/b2to2btyYy3oQ+VwiAABp5JP3A0Axy7NHTnaXpXYZud1u/fg9sbn3yXxRTbrAplaWg0vlLbsUjUaFEA6Hw6g/PDyctlTmD8Z7RdbZoSck7f8Lmf6ziMfjSZm20cGYyx5Sb+I9Hs+kI6fXvTQV6skamnmmZm40beO85AMyqRPFp+4h90vEhNDrDqBkcaMDYNbK87ZeZKanu08d954+U+XUjePeW6uqKlMLuSZTpqQ3+ylkqp/LRmlsbMzj8cikwvwU8bh7SL37z5QP5IjUvTTl/3unmWfZaGxP/Woj9XuEtHvI8RIxIaTuAEoWD8wDwMzT3Nzc1tYWi8Xk/M8rV64syG5ln5j5UVj5OnW8rjR//vzm5mZd1+U8VXL6q1yeiW1oaBDpZpg3pq0GUAzNXG5MnWpOPjw/rklfIgAAqUjdASA9+bCrubvJUKidm9c3losqy+1Saqlxb33ZZZcJIUKhkMx1Kyoq0h4lyxe3aeuXl5cLIcbGxowtzz//vBBCrguVRE6abU4ADh8+LIQ466yzsp15VjnmA0Ch0MxF1mYuK5uDlE3efAqZTMUlAgBKGak7AKRXV1cnhHC73bFYTG4JhUKKonR2dua4hywzOcmde71eeU88Ojra09MjhLj66quNOqmla9askUU2m83n88mJozJ1iU/C0qVLhRA9PT3GceVEWWm7++QcVH19ffLHkZGR/v5+8cZ1pDKR3W6hUMj4iOSiWXTHYZrRzLM3c9mcvV6vcZpy2jnzKWSSzyUCAJDGxJ6vB4CZYyoWh1NVVU7XlHoJNW8xJmcyz0c17s6TVo1KqmCUSsYyS4ODg/mcY5LUOd7lKaSeY+ocVCLDYPXU0x8bG0t9r/HZTgJj3UvTFC0ORzM3n4ichyJT5Uzv0idyiZgQxroDKFnc6ACYtQpyW+/z+YxHQz0ej5FbZr+nD4fD8l1yMqe09/S6aalkYw6npF3JXmi5lFrq22W1Sae7aclppYyoPB6P+VnipBMx5qCSKYd5GerUOJM2xuPxpNNP+9ByjkjdS1Ohfu808yzNXNf1wcFB40wz5d5pTz/HS8SEkLoDKFmKXojRXABQhPr6+jZs2DATr3KKogjTHNRpjYyMLFmyxOFwbNu2bbriKl4z93eNfMzo3zvNfHLkMAT57D0AlBTGugPAjCSHxeYyWRSAGYpmDgAwzLE6AADAxMjOOiGE0+nMNOk0gBmNZg4ASEKvOwDMMHKEqtvt7ujosDoWAFOCZg4ASEKvOwAUnezDXwOBwLRFAmCK0MwBABNCrzsAAAAAAEWN1B0AZgZFUYzhrxZKJBJer1cG097ePjIyUqhSAKJoWroQYmRkpL29Xcbj9XpjsZhRpKSTugdN04rkXABgFiB1BwBMQFNTU0tLi3ztcrmWLFkSiUQKUgqgeEQikSVLlrhcLvljS0vLxo0bE4mEEGJ0dDTHPdjt9ikMEQBKDKk7ACBXfr9f0zSPx6Pruq7rwWBQCNHV1ZV/KYDikUgkKisrVVWNRqO6rsfjcbfbrWnarl27jDput1t/I/MehoaGKisrpz1wAJjNSN0BALnq7e0VQqxfv17+WFNTI0zpdz6lAIrHs88+K4RobGwsKysTQthsto0bN4rjrfjw4cNCiGXLlmV6e2dnZ3V1tc/nm6ZwAaA0kLoDwPhCoVBra6sxSNv8mHckEuns7JRFdrvd7/cbRcb4Tzng0263a5omi/x+vyxNW99cKp9QzR6V3W4PhUI5BmyWdsBqlqG2gUBA13WbzSZ/lKdj3KDnUwoUA1q6tHfvXiHEqlWrjC02m03X9RznvW9rawsEAvX19blUBgDkSgeAWWr79u0FucqlvVsNBoOZinw+n3yj/DGpTjgcdjqduddXVdVcakSVtBMhhNPpHDfgJJn/cxjnc3O73UnBF6p0cgr1u8bMUtjfOy3dIFeV13Xd5/MZK8yPjY3JUtmEw+Gwx+ORO/F4PPF4PNNxJ/6ryKa2tra2traw+wSAGYEbHQCzVqFu6+Xdpxzzqev64OCgEMLhcBhFg4ODsigajZpvVY37bHlTK0d3p91irm+ML41Go/KuXd6Lm2vKNxr7icfjsmY4HM4ecKEEg0G32y3v6VMz8HxKJ4fUvTQV9vdOS0/6KJK+NVBVVYZhfPuWtjR1P/nHY0bqDqBkcaMDYNYq1G290ekUDAbT9iyNjY2Fw+FAIGDc6crt8rXRVWVsMe8ktf7w8LBRKjMEc/Igt8sDmfcTj8fF8Ymjxg24gGS3W9qOvjxLJ4TUvTQV9vdOS0+K1ul0Giclx7bI79pkqfFFRlJp6n4KGJhO6g6ghHGjA2DWKtRtfTgcNvcsJaWaqY+zJt2gmytn35L2NtfYmFozrXEDTt15pv3kQiYSxrO+BSydEFL30lTY3zstPal+6lcGWVpr2tIJXUxyROoOoGQxTR0AjKOiokLX9XA4LJdHWrt2rd1ul9NBeb1el8vlcDiCwWA4HB4bG7M6WCGyBlxwcto5Y1KuApYC04yWbpDfUxiTSorcWittGQCmlrXfHADA1JmKnthoNGoetpp0IZUdUyK/vjhj5Kqu68PDw+L4rFTmmg6HQ7yxTyzHgPMkH9A1H1fmMPJB33xK80Sve2maut97ibd0+QC8OUJ5ypnasrnUbCpuNel1B1Cy6HUHgHHItZeGhoaEEGVlZYsXL06qMDIyIoRIJBJpZ2+aKK/XOzo6KoQYHR3t6ekRQqxZsyapTl1dnRDC7XbHYjG5JRQKKYrS2dmZS8CT1tjYKITo6+uTPyYSCRmhjCefUsBytHSDXBbO6/UaS9bt2rVLCHH11VeL421ZbjGX0pYBYGpZ/d0BAEyVQvXIyXmbk3g8Hv1431QqOQFV6mU2+xb5OmlIrbEQVNJ7U0feqqoq55TKEnD+ZIdb2gjzLM0Hve6lqbC/d1q6WeopG53q8Xg8x7ac+jnkj153ACWLGx0As1YBb+vNSzQ7nc5AIGAUGSsbO53O4eFhOXGUvHue3A29fnztpaR5p1Lf6/P55PO08ojmCa6zBJw/Y6lnOfS3gKWTRupemgr+e6elmw0ODsrjqqqaNHt8PB7PpS2TugNAASl61nlHAWDm6uvr27Bhwwy6yimKIkwDa5G7Gfe7RkHM0N87LT0f8rH8/v5+qwMBgOnGWHcAAAAAAIoaqTsAAAAAAEWN1B0AAAAAgKI2x+oAAAD/wNhXoBTQ0gEAk0CvOwAAAAAARY3UHQAmSVEUOVN0MYSRNhhN0zJFOLmiTBKJhN/vt9vtiqLY7Xav1xuLxfIszXJewPQokj+/1LagpJO2fpZGNImWnv29iUTC6/Uabdnv9ycSiXFLaekAkCvr1qUDgKk11Wt9F8lVNNMlXa47nTbCyRVlEo/H5fLOZqqqytWnJ106of+qWNe9NJVmG49Go1nu6LKXGibR0sd9r7H4vLktj1s6oZaus647gBJGrzsAzAbymm78ODQ0VFlZmbbm5Iqy2LVrl6ZpHo8nHo/ruh6Px51Op6ZpPT09+ZQmnRFQ4pJahNvtTrqlM1fOXjq5lp79vZFIpKury+l0RqNRXdej0ajD4dA0bWRkJHspLR0AckTqDgCzTWdnZ3V1tc/nK1RRdr29vUKI5uZmm80mhLDZbG1tbUII+W8+pQBSHT58WAixbNmySZSKPFp69vfu27dPCNHU1FRWViaEKCsrk93sBw4cGLcUAJALUncApSuRSCiK0tramrS9tbVVURQ5DjMSiXR2dspxmHJ8ZtpdpQ7UTN0SCoXknu12eygUyhRV2nGqExoI2tbWFggE6uvrC1WUXSAQSOo0k3l4/qVAnmZrG8/HpFt69veOjo4KIRYsWGBsWbhwoRDi4MGD45YCAHJS6CfwAaBY5DIO1u12CyHkuGtJjrKWj5sGAoHUy6bP55M1zVfR1Ctq0han05m0H6fTmTakiV6xp7MoR8PDw+YPKp/SHINhrHtpKs02Ls8oHA57PB5ZZAw5Gbc0y24nZNxPI2lj9tIJBcNYdwAli153ACXtiiuuEEKY+8fkazl3mt1uF0IMDg7KK6ac/6mhoWGiRwmFQi6Xy+l0mkd0u1yuSCSSWjnLJXuyZzndenp6VFVdt25dwUuBiZqtbbyysrKlpUW+bmlpaWpqMk/nnr0UADATkboDKGkVFRWqqsoR11Jvb6/D4TjvvPPE8Tvss88+OxKJaJrm9Xond5RHHnlECNHW1pY0ontgYKAA51Bk2tvbXS5XR0dH2kff8ykFJmH2tXG5Z+PrBl3XfT6fpmm7du0atxQAMINNtJseAGaKHB+iDgaDQojh4WH9+NPawWDQKE19CFake8Iz9YqaWprjRXiiV+zpLBqX/LjC4XChSnMMhgfmSxNt3FzHvBJbLqX5tPTsn0bqxuylEwqGB+YBlCx63QGUuuXLlwsh9uzZI45Pdyy3CCG8Xq/L5XI4HMFgMBwOJy02DrNYLNbe3h6JRIaHhysqKgpYCuSpRNq4pmmTLs2f/AbE/Fi+fC23Zy8FAOSC1B1AqbPZbB6Pp6WlJRaLNTQ0eDwe42ltOVh027ZtNTU1FRUVc+fOzXGfsVjM/KNcBinLTFFmWb5tzes8p1IkEtm4caMQoru7Wz6HXKhSIH+zrI3b7XZjenxJvpYxZC+dOuXl5UII83cfzz//vBBCrgaXvRQAkAtSdwAQq1evFscXLrrqqquSSkdGRoQQiURCTt2clpzyamhoSNa86667zKV1dXVCCLfbbdzuh0IhRVE6OzsLeBZWGR0draysrKio6OjomD9/fgFLgUKZTW28sbFRCGEeuy5fyxiyl06dpUuXCiF6enrkOnCjo6M7duwQQqxcuXLcUgBATib6hD0AzBQTGv8su6QcDod5o8/nS3vllINmzVfRpJpGAmDsKvW5UFVVzQtWTVqWi3mhirL/f2GsQZX2v5h8Ssc9tIGx7qWpNNt4PB6X3yOYGQvRZS/Nstu0W3IMSUo9rvnTzl6a+6EZ6w6gZHGjA2DWmtBtvZzIyjwts2Tklk6nc3h4OBwOCyE8Ho+ecq/p8/nkvWnaUlnBeGbV4/EU5J4+7YEKXpT9rlpklmfpuIc2kLqXppJt4/F43AhGDtTPvTTLbnNsbplqjo2NeTweeVxVVZPWk89emvuhSd0BlCxFL+LBkwCQj76+vg0bNsz6q5yiKGK8NLggR7Hkk8zx7Erkd40kJfJ7n542bhyrmFu6OP7kf39//5THBABFhrHuAIBxDA0NZXmyHcDsQEsHgGJG6g4As4GiKLLbairs3bu3ubl5inaeyZSeETDjTEOLoKUDQDEjdQcAjGPTDhylbAAAIABJREFUpk1WhwBgytHSAaCYzbE6AABAXmbrQN/Zel7ARM3utjC7zw4ACohedwAAAAAAihqpOwAAAAAARY3UHQAAAACAokbqDgAAAABAUSN1BwAAAACgqDHDPIBZjhWDgdmNNl5qqqurdV3n9w6g1JC6A5idDh06dP/99yuKsnTp0ttvv33OnFK53A0ODn7zm9/cvn271YEAU27VqlUl+6e+YcOGW265pbq62upAppvX6w0Gg1VVVVu2bFmzZo3V4QDA9FFYThPALPPiiy9+7Wtf++Y3v/nud7+7o6Ojrq7O6oimVV9f34YNG7i2A7Oboijbt29fv3691YFY4JlnnvmP//iP/v7+K664YuvWrRdddJHVEQHAdGCsO4DZ45VXXtm6des555zz7W9/e+vWrc8880yp5e0AMOu9973v7evre/zxx//2t7+9//3vX79+/aFDh6wOCgCmHKk7gNng9ddf7+/vLy8v7+jocDgcR44cufnmm0vnIXkAKDUXX3zxY4899vDDDz/33HPl5eU33HDD7373O6uDAoApROoOYMYbGBhYvnx5fX39pZdeeujQoS1btvzLv/yL1UEBAKbcFVdcEQ6Hv//97z/88MOLFy/evHlzPB63OigAmBKk7gBmsIMHD15zzTVXXnnl2972tnA4fP/99y9cuNDqoAAA0+eEE06oq6sbHh7+xje+ce+9955zzjlbt279v//7P6vjAoACI3UHMCP95je/ueGGGyoqKmKx2E9/+tPdu3dfeOGFVgcFALDGySef3NLScuTIkdtuu+0///M/zzvvPI/Hc/ToUavjAoCCIXUHMMO89NJLmzdvPvfccx988MF77733ySefXL16tdVBAQCs9+Y3v/nzn//8kSNHrr322ptuuunCCy/s7+9nxQ0AswOpO4AZ47XXXvN4POeff77X673jjjtGRkauu+46RVGsjgsAUETe/va3b9myZXh4eMWKFRs2bKiurn7kkUesDgoA8kXqDmAG0HW9v79/6dKlN910U319/ZEjRz7/+c/PnTvX6rgAAEXq3e9+9/33379v3743v/nNNTU1V1555VNPPWV1UAAweaTuAIrd4ODgJZdcUl9ff9FFFz333HN33nnn6aefbnVQAIAZ4P3vf//u3bt3794dj8eXL1++fv36w4cPWx0UAEwGqTuA4vXcc8+tX79+1apVp5566v79+/v6+t7znvdYHRQAYIa54oor9u3bt3379kgkcsEFF9xwww2///3vrQ4KACaG1B1AMfrDH/5w8803X3jhhQcPHtQ0bffu3cuWLbM6KADATKUoSl1d3cGDB7/1rW/9+Mc/lovAJxIJq+MCgFyRugMoLi+//PLWrVvPOeecH/7wh3fffffTTz99zTXXWB0UAGA2mDNnTktLy6FDh9rb2++55x65CPzf/vY3q+MCgPGRugMoFkePHvV4PIsXL3a5XLfeeuvIyEhLS8uJJ55odVwAgFnl1FNPlWvIbdy48Utf+pJcBP7YsWNWxwUA2ZC6AygKAwMDy5Yt+8xnPmO3248cOXLHHXfMmzfP6qAAALPWW9/61i1bthw6dGjdunU33ngji8ADKHKk7gAstm/fvssvv/yDH/zg0qVLf/nLX95zzz3z58+3OigAQEk466yz7rnnnmeeeea9733vhg0bVq1atWfPHquDAoA0SN0BWCYajV533XVVVVWvvvrqo48+2tfXt3jxYquDAgCUnCVLlvT19T355JOnnnrq5ZdffuWVV4bDYauDAoA3IHUHYIEXX3xx8+bNS5Yskav1yJXbrQ4KAFDSVqxYEQwGd+/e/ac//UkuAn/kyBGrgwKAfyB1BzCtXnnlFTmB/Le//e2tW7c+88wzdXV1VgcFAMA/XHHFFfv37/f7/eFweOnSpTfccMPY2JjVQQEAqTuA6fL666/39/eXl5d3dHQ4HI4jR47cfPPNc+bMsTouAADewLwIvKZp55xzzubNm//85z9bHReAkkbqDmA6DAwMLF++vL6+/tJLLz106NCWLVv+5V/+xeqgAADI6KSTTmppaTl8+DCLwAMoBqTuAKbWwYMHr7nmmiuvvPJtb3tbOBy+//77Fy5caHVQAADkxFgE/l//9V+/9KUvLVmyhEXgAViC1B3AVPnNb35zww03VFRUxGKxn/70p7t3777wwgutDgoAgAmTi8CPjIx86EMfuvHGG9/3vvf19/dbHRSA0kLqDqDwXnrppc2bN5977rkPPvjgvffe++STT65evdrqoAAAyMs73/nOe+655xe/+EV5eblcBP7RRx+1OigApYLUHUAhvfbaax6P5/zzz/d6vXfcccfIyMh1112nKIrVcQEAUBjnn39+X1/f4ODgKaecsnr16iuvvDISiVgdFIDZj9QdQGHout7f37906dKbbrqpvr7+yJEjn//85+fOnWt1XAAAFN4HPvCBUCi0e/fuF1988aKLLlq/fv2vfvUrq4MCMJuRugMogMHBwUsvvbS+vv6iiy567rnn7rzzztNPP93qoAAAmFpXXHHFz3/+c7/ff+DAgfPPP59F4AFMHVJ3AHl57rnn1q9fv2rVqnnz5u3fv7+vr+8973mP1UEBADBN5CLwzz777F133RUIBBYvXswi8ACmAqk7gEn6wx/+cPPNN1944YUHDx7UNG337t3Lli2zOigAACxgLALvdDq7urrkIvCvvvqq1XEBmD1I3QFM2Msvv7x169Zzzjnnhz/84d133/30009fc801VgcFAIDFTjvtNGMR+DvuuINF4AEUEKk7gAk4evSox+NZvHixy+W69dZbR0ZGWlpaTjzxRKvjAgCgWJxxxhlyEfirrrrq05/+dEVFBYvAA8gfqTuAXA0MDCxbtuwzn/mM3W4/cuTIHXfcMW/ePKuDAgCgGL3rXe+Si8BfcMEF69evv/jiix977DGrgwIwgym6rlsdA4Bit2/fvttuu+3RRx+tra398pe/vHjxYqsjwj/99re/ve66644ePSp/TCQS0Wj0fe97n1Fh6dKl27Ztsyg6AIXR2tr67LPPGj8+/fTTixYtstls8sc5c+bcf//9Z555pkXRYRxDQ0ObN2/es2fPFVdc0dnZab5EA0CO5lgdAICiFo1G29vbv/e9733gAx949NFHL7nkEqsjQrIzzzwzGo0ePnzYvHHPnj3G69WrV097UAAKbP78+V1dXeYtL730kvH6nHPOIW8vZlVVVT/96U8HBgY+97nPLVu27OMf//hXv/rVd7/73VbHBWAm4YF5AOm9+OKLmzdvXrJkyb59+7Zv3z44OEjeXrSuu+66k046KVPphg0bpjMYAFOhoaEhU9FJJ530yU9+chpjwSQZi8D//Oc/X7JkyQ033BCLxawOCsCMwQPzQInq7e1duHDhmjVrUoteeeWVu+666ytf+cpJJ53kdDpvvPHGOXN4QqeoHTly5Nxzz029niuK8t73vvfpp5+2JCoAhfW+973vmWeeSXvnNjIycu65505/SJicv//97/fdd98Xv/jFV1555dOf/vTtt9/+5je/2eqgABQ7et2BUjQwMHD99dd/9rOfTboFfP311/v7+8vLyzs6OhwOx5EjR26++Wby9uJ3zjnnVFZWKoqStH3OnP/P3pvHR1Hk///VOYRFNLq6sCy7rAfKKghh/YgGRQRFkGUiDyEJt8cGPgkKfFfjgTsIGq/FiQsCD3DigUSZJLjshxnXLOrMeoAJh2GiAiawfEgE14nXjCJnkv79UT/60/ZR0z1X9cy8nn/wCFU11e/uetW7qrrryLr99tu5mAQAiDmzZs1Sn+ghCMLQoUMxbk8uzjjjDHoI/J///OfVq1fTQ+BPnjzJ2y4AgKXB0B2AtKOxsfHWW2/t6ur6+OOPXS6XFP7OO+9ceeWVU6ZMGTFixL59+55++umzzz6bo53AFJp9+o6ODsyWByBlmDZtmvqE8MzMTLyhS1J69uxJD4G/6667Fi9eTA+B7+rq0kzc0dGxbt26BFsIALAUGLoDkF4cOHBg7NixJ0+epJ2DBx544MSJE7t3754wYcKYMWPOP/98v9+/bt26Pn368LYUmKOoqEjR4cvIyMjLy/v1r3/NyyQAQGz51a9+lZeXl5Hxk85bZ2fn5MmTeZkEouf888+nh8DffPPNc+fOzc3N1TwE/sUXX7zjjjtefPHFxFsIALAIGLoDkEZ8/fXXY8aMCYVC9CAxURS//PLLcePGDR48+Ouvv37//ffffvvtK664greZIBL69OkzYsQI+Yd3QRBmzZrF0SQAQMyZOXOmfGlMRkbGyJEj+/bty9EkEBP69ev3/PPPNzU1/e53vyssLBwzZszOnTul2KNHjy5atEgUxTlz5vz973/naCcAgCMYugOQLhw9enT8+PGff/75qVOnpMDOzs5t27ZVVlbW19ePGDGCo3kgembOnKkIwbc4AFKMwsJC+dBdEAR1xQfJy8CBA2traz/88MOTJ08OGzassLCwpaWFELJ8+fJvv/2WECKKYmFh4VtvvcXbUgAABzB0ByAtOHXq1MSJE/1+v3zcTuno6Ni7d696hzOQdEyaNEmaSZuZmTlmzJjzzjuPr0kAgNjy85///KabbpJ2DxUEYeLEiXxNAjEnLy/vvffe27hx46effjpo0KA777zzqaeeotsciKLY1dU1ceLEjz76iLeZAIBEg6E7AKmPKIrFxcU+n089bieEnDp1atmyZQcOHEi8YSC2nHPOOePGjaN9elEU8S0OgJRkxowZdGOLrKysW2655ec//zlvi0BcmDhx4ieffLJmzRqfz3fs2DEpvKur6+TJkzfeeONnn33G0TwAQOLB0B2A1OfBBx989dVX1fsSy/nzn/+cMHtA/Jg+fTot6Ozs7Pz8fN7mAABiz8SJE8844wxCSGdn54wZM3ibA+JIZmbmuHHjAoEA3aFGorOz8+jRozfccENbWxsv2wAAiQdDdwBSnGXLlj3zzDN6h82cccYZWVlZHR0dtbW1n376aYJtAzHHZrN1796dEHLrrbf27NmTtzkAgNhz5pln0hdz3bt3nzBhAm9zQHxZtGiRZgt+6tSpb7/99qabbvrmm28SbxUAgAsYugOQylRXV997773Sf7OysqQdyH/+859ff/31JSUlq1at+uCDD77++utBgwZxMhPEjB49etx2222EEHyLAyCFmT59OiFk0qRJPXr04G0LiCPNzc1r167VXOxGCDl16tTBgwfHjBlz5MiRBBsGAOCCIIoibxsAN2pra4uKinhbARLB+eefP2jQoMGDBw8cOPDyyy8fOHDgueeey9sobkD5yQ5aLqAHdtxMSWpqagoLC3lbwYFJkyZt3LjxjDPOOHnypF6arKysG2644c4776QvdECKgfYOyMnibQDgT01NDW8TQOw5fvz4qlWrtm/fvmXLlssvvzydB+p6pKryOzs7X3/99VR9N1FfX79s2TLeVgBL8//+3//Ly8vjbUXccblchYWF0kSqFCZVvZkR5s+fP3bs2Obm5t27d+/evfvw4cOiKAqCcMYZZ3R0dNCdTTo6Orxe748//khSt11LT9DeATUYugOSnm+y04Hu3bsXFRVde+21vA2xKCms/Ntuu61bt268rYgX6MoANnl5eSlcuyVuvfXWFK7mctJ56D5y5MiRI0dK/z1x4kRzc3Nzc3NLS8vevXs/+eSTffv2HTt2TBTF+vp6ktLtWnqC9g4owNAdAABSjTTp0AOQzqCapyHdunUbPHjw4MGD5YGHDh1qbm52uVwvvvjiv/71r1GjRvEyDwAQb7BNHQAAAAAAAEnJr3/96xtvvPHmm28mhGDcDkBqg6E7AAAAAAAAAABgaTB0BwAAAAAAAAAALA2G7sAQoVCouro6Pz9fEIT8/PzKysr29vaIcxMEIcrze6LPgTuhUKiyslJ6pNXV1aFQSC+xoEUirQVpgsfjUUuLapWqbtGiRS0tLYwcTAkbAF6YUrUmaMgIGrLkR9Pnx5sUKHooH3BDBGkMPUQkbLJgMGiz2RTKsdlsgUAgsutGr70UUG9JSYn6kWqmbG1tjazyGizfNARPRhO/368pLXX19/v9epkYF3ZkoOwAG0JITU1N2GSmVK13ITRkCWjIRMNlmuZE4Bv1fL4eDocjJoqF8g3ePto7oAZf3UF46urqPB6P0+kMBoOiKAaDQbvd7vF4qqqqeJuWrDQ1Na1Zs8Zut7e2toqi2NraWlJS4vF4GF9+HA6HovYm0mCQ8jQ0NOTm5qrDq6urafWnqvN6vYSQNWvWaGYSgbABSDymVA30QEOW1Oj5fAZlZWVxMia5gPIBRzB0B+FZv349IWT27Nk5OTmEkJycHOq+09yJNzU1VVRURPbb7du3E0JmzpzZr18/Qki/fv3oG9zGxkZ14v379xNChg4dGrmtINWJRo2EkIqKiry8PJfLpY6i1V86K3j06NFEf5BjStgA8MKUqlMbNGTpCcPnpwlQPkhSMHQH4XG73YoXhHQMT6GLdtrb2ysqKqQ1P4ocpHXy6ig28jX2paWlem80qQumlqgN8Pl8paWl0prGpqYmI1EMGhoaSktLc3NzpZcXmguZGMuZ2traCCG9e/eWQvr06UMI2b17txEDgDVh65DIKgJdW6tQiKTG/Px8n89n8KLRq5EQUlZW5na7p0yZoo6i1V+q8h6PhxCi1+GDsEFSwFY1rSxtbW35+fmLFi2SfoWGTAHqe/LC8PlER1GSDNitiRwoH4DYE8PJ9yDpiHgVTXNzMyHE5XKJp4f0inWDNIqiWBFE10oZvK7eckR5Dm63Wy1syQDNWK/Xy47SJBgMut1ualJJSYnb7ZZW+5utYppReunpE/P7/U6nk6aRFi+wwSopPeLxZNg6FEXRbrczFKKOtdvtjMvFUI1yGMmkmiu/KSM/N3hpg0DVgA0xsy5aU9VSBZSHoyFTY6q+R9yQiVjrbozIfKO6vPQUZUQSCqB8MTrlo70DaiCItCZip2C32202G3U91BPZ7Xb6X+rfpe066H/lK4KkIUrYq1C/LOVMP4mUlJSIP3WR9O/6+nr6X2lHEHksvbooivX19YpMNKMUtLa20qvbbDaXyyX9JGIi8PsKpOfPAE5fj3g8GbYO1RVBGgnIY+XbSRCdfbNirkb1XWhGeb1eh8NB+z16o3dTwo4MqBqwIWaGeZqqpoqVixwNmSaJachEDN2NEauhe1ixGcwZyqdEo3y0d0ANBJHWROYUFIMK6obku83L/RdNLHeU9Iu9kevSsY3mPvbqHAKBgN/vd7vdih4V7ZM5HA6v16twlIwozcsxPjaaxZTfVzRs4ukmMKw9cPp6xO/J6OmQXRForFyEwWCQaG1sI8ZBjerM2WnopwPNjxumhB0ZUDVgQyIa5slVrW7U0JAxMjQSKEbRkIkYuhsjVkN3hqJMOXMoXx4emfLR3gE1EERaE4FTUH8MVHsreUg0XXlGMkWU5lRkGiWdfUIIsdls8vEGI0pB2Fe26qsrzDBya6YaRWLgzC04fT3i9GQYOmSXuCn9xFyNmiYxoK8VNOUXvbDDAlUDNiSiYZ5c1WrFRiNs4w0BGjJGMgzdwxKroTtDUWYLF8rXw6Dy0d4BNRBEWmPKKQQCATpPvrm5WR6u9lbykGgcnEG/Sb+WlJSUeL1ev98fCAQ0myJpzpLNZpO/emBEKYjhQim9r6zs5c16T0APOH094vFk2DpkVwSz+hF5rHU3kix6YYcFqgZsSKTDPEV91IwKG8jIlh2FhoydDEP3sMRq6E7RVJTB8mLnLEL5hp8k2jugBoJIa4w7Bb/fb7PZ7Ha7eu6T2gHJQ6iDk4/2FQuZGBicbaXIjTpQzfxbW1ulrVaMR6mpr6+XL1SOAPoCWP5Y6Ptj6ZxhObSlUTcSmmu65MDp6xG/te7SfxU6VE+4lVcEKieD+9YoiF6NctT5qOVHu1aa8jMl7MiAqgEbYmCYx1a1uhagIdMkMQ2ZiKG7MWI7dKcoFGVKLVA+JRrlo70DanA4HAhPW1tbbm7ukCFDysvLe/XqZeq3o0aNIoSUlZXRszTa2toqKysN/nbkyJGEkBUrVoRCIUJIdXU1PV9EMzE9dCQUCil2BKEHhzQ0NBBC+vXr179/fyNRbK655prVq1fL3/Wa5bLLLiOEVFVVSY/l9ddfJ4QMGzZMnXjatGmEkLq6OimE/l1QUBDZ1UH80NMhrQiVlZWaFYEWpcPhaG9vpyE+n08QBCOnzkavRjZUfrW1tfS/oVCoqqqK6MjPlLAB4IUpVRM0ZDqgIUtJwiqKKjksUD4Fygcxhve7A8ATg+/zpAMtNPWjFpIiRLGKScrNiIVGThbRO2KavhClm44qoG9GGVEJQH1r8lew8hsMBoPqxEbmZeF9rR7xeDJsHYrmD4ez2Wya3yviitwkCbb8FD9hCzt6oGrAhhj7QstQtWYtQENm8NZi3pCJ+OpujFh9dWcoSipBg14dyhejUz7aO6AGgkhrDDoFTa8qOSa5h5Knl4e4XC756TvqBHoEg0Gph2S326VRkCIHRRrFzCW/3y/1uux2u9vtln7IiIo3gUDA6XTSx2Kz2RTnfCpuMBgMSs+QLgkzcgk4fT3i9GTYOhRlFYEmUJSyy+WSZvE5nc7Ej9tF/brJkJ/iJ2xhRw9UDdgQw8M8PVWHrQVoyCQS0JCJGLobI4YT5vUU5ff7aSNlZIs1Eco/TcTKR3sH1Agic2AGUpva2tqioiJoIFVB+ephkScjCEJJScnq1av5mpFcWKTsgGURBKGmpqawsJC3ISBmoEyNAN+YeqBMgRqsdQcAgLgjCIK0Ko8QEgqF6Dp2uhoQAAAAAAAANlm8DQDpiyAI7AR40QhSBrfbnZ+fn5eXJw+02Wy33HILL5MAANGDhgykJ1A+AFzAV3cAAIg7NpvN6/VKq/JKSkpcLldVVVVOTg5fwwAAAAAAQFKAr+6AG3gjC9KK0aNHjx49ury8nLchAICYgYYMpCdQPgBcwFd3AAAAAAAAAADA0mDoDoBRQqFQZWVlfn6+IAj5+fnV1dWhUIidmG5OtmjRopaWFilK0EedT1NTU9gVZQDEEFM6J4RUV1fTxKWlpU1NTXrJ1EoOhULSb/Pz8ysrK9vb22N2GwAAfeJRzT0ej7q1UlTzsBcCIN6YEj+jqxa2L2ewmweAObgcSQcsApcTIx0OR5IKTzpzW4JxtCk9wFOO3++nUXqVUZ1bIBCIpp7iRFA9EvBk0lbn9MhrBWolB4NB9W9tNlvY0+yhasCGJPAMcFRzCXoWt+JpBAKByKq5gkSWafKSYN+YDuJvbW1Vd9Wku9aMknJj/9YgaO+AGggireHiFCJwXlaA9kvsdntra6soiq2trdT7Nzc3qxO7XC5CiNPppP/1er2EkJKSEnbm6qykBiYym+H09UjAk0kfnTscjmAwKA+hv5WjVrJUR+hvg8Eg3cPP4XCwLYSqAZtEDvNQzSn19fWaIxOn00l+Os5XNI4GwdDdCAn2jekgfjr8DtsqKTKnWZn9rSZo74AaTJgHwBDbt28nhMycObNfv36EkH79+lF339jYqE68fv16QkhhYSH97+jRowkha9as0cy5vb09NzfX6XReeuml8vCKiorDhw/H8h4ACEcEOi8uLpb2yadn3W3evFmeTFPJ9LezZ8+mv83JySkrKyOE0H8BAPEjttW8oqIiLy+PjskVzJkzhxAyZcoUKYT+TcMBSDymxL9//35CyNChQ43krOjLmfotAMbB0B1Ejs/nKy0tpYuFfD6fFE7X87S3t1dUVEhLiaQoeRrpj7a2tvz8/EWLFkmZKFbHKTKnCejf0jql/Pz80tJSuYWhUIiuzVMbb2rBOSGkra2NENK7d28ppE+fPoSQ3bt3qxO73W5RFKWOjsfjIYRo9mwIIStWrLDZbLNnz5YH+ny+srIy7EZuBaBzoqNzKmz5+Xb0b3kfSE/JtI7IQ3BOHuAIqjmJtJqXlZW53W75+FxCPVueHQ64APETHfGbQrMvB0Ds4f3ZH/Akmqk40gnVEna7nUbR/yraZjplTi0/6bdENq0ubOZut1seS1cW0Xnp0pJy8fQsPjpIUGC2UmhGha1EdDEY0VkALNns9Xrlgc3NzYonxrgEA0y10sP4k4HO9QLF0ytgpWm0oigGg0F5YlNKlidmAFUDNsT85GpUc71A0UA1Z+RAb009YV7zLhhEUKZpSGS+EeLXCxRPd+H8fj9d+kFki7wUqPtyxn/LAO0dUANBpDUROwXqpOx2u2KdKnW1koOmsTSxtAuIwkXKGwNF5tJKJJo59YmSf9eMtdlsUsMgnnadmkuYzGLK3cvvxeFw0JZPc0xis9kU+6PQ7bsUzVtkNsPp62HwyUDnjEDxdC9cuinp+dDEZpVst9ttNlvYng1UDdgQk8M8VHNGoBiumofNwev1ysd+NptN8araoMEYuoclAt8I8TMCRdnXFzma7ZS6L2f8twzQ3gE1EERaE7FToB5W/RqebshBPZR8C1m5W9R09/LEdN2RfAscutsH3eaNppd7cHksbSqk3xLmrrmmMOXu1dDXrooui+aLZPpspQdi/BJq4PT1MPhkoHNGoKi1S7y8T29KyfL+IhuoGrAhJod5qOaMQDFcNQ+bg3oAY3aPOhFDd2NE4BshfkagFF5fXy+FKN5kUTT7cgZ/ywbtHVADQaQ1ETsFoo+o5QTDunu9xOrAsG7XZrPRVodu9anXS2Dfgl56I4Ga0OZQ0fbQhk3ealLPLvf1xi+hBk5fD4NPBjpnBFICgQB9LWWz2eQTKU0p2fi4XYSqQTiIyWEeqjkjkKJXzcPmoB6rRDB6ETF0N0YEvhHiZwTqQQz05Yz/lg3aO6AGgkhrUnLoTpdOBQIB2kXQGw+Ydfd6L6flc7rYKDKnJ10rfs6wKoKSgtPXIwWG7tbUuXQcjkElBwIBOk/e+ExIqBqwISk0dLd4NWdYrhciRvQwi3AoAAAgAElEQVQF1WyZpicpNnS3pvjVlmv25Qz+Nixo74Aa7DAPIoHxijFWmdNdQCktLS1SOEUdK03hu/766wkhPp+PHmkzZMgQzaswaoVm+oEDBxJCqI+mHDx4kBBCzxdRQPdTpXulUtrb2xW3cODAAULIsGHDNC8HrAB0TkzqnB6H07dvX9adn6apqam4uJgQ8sILLyhORgQgYaCak3hWc03orvWAOxA/MSl++reRvpyR3wIQARi6g0goKCgghDgcDjoiJYT4fD5BECoqKgzmIHdnmplXVlZSn97W1lZVVUUIGT9+vJRGHTtq1CgalZOT43K5pk6d6vF41LubRsxll11GCKmqqpKu+/rrrxOdsfe0adMIIbW1tfS/oVCIGklvjfLJJ58QQgYMGCD/oV7bE6umFJgCOjel85aWlg0bNhBChg8fHlbJbW1tubm5Q4YMKS8v79WrV6zsB8AsqOYRV/OwF6ITcHw+n/SI6PFgYSfmgMQA8RsRf11dnRRC/w7blzP4WwAigfG+CqQ8sT0czmaz0R1K1NKSh0gb3si3KgmbuWKfakUCxVQl6ega+VLb6FEfRUtvQX2PmokVRtI3r/I9XTSJpp5iqpUe0RwOB51LN6Lev4ror2JVPAHpvBw1bPOgasCGxOJwOFTzCKq5+vYDgYD6t9KzNU4EZZqGxOpwOIifLX6DfTkjvw0L2jugBoJIa6J0Ci6XS5r543Q6GVtJy0P8fj/9FV3tpunuaebU60n74iiyoq/t9U6aocnM9g/Y0K16JKsUR3Sqb0S6hZKSErWRejceWTJN4PT1MPVkoHOGzqX9q2inhLHVnOKHRB+2eVA1YEMiGuahmkdfzTVvPxgMKm7f7NHWIobuxojYN0L8DPHLBWy2Lxf2t2FBewfUCCJm4aYxtbW1RUVFyaUBQRCIbI8TTVpaWgYMGFBSUrJ69epE2WVFkrF8E4P1nwx0rof1yw7wRRCEmpqawsJC3oaEB9XcIElUphxJLt8I8RshucoUJAasdQcpCF0xhb1AQGoDnQOQ8qCag7QF4gdATRZvAwCIJfQ9LiHEbrfr7UcKQLIDnQOQ8qCag7QF4gdAD3x1BykFXVPkcDjKy8t52wJAvIDOAUh5UM1B2gLxA6AHvrqDJIO95sftdifMEgDiB3QOQMqDag7SFogfgMjAV3cAAAAAAAAAAMDSYOgOUgFBEKSVUVbA4/Go7QmFQtXV1fn5+YIg5OfnV1dXh0IheWxlZaVeLAAkSXQuaCHFUp3TwEWLFrW0tCTWZACSD4tUfEUTVllZ2d7erk6m6RYAMIJFpE4IaWlpWbRoEbVHIXV2K4a+HIg7HA6kA5YhZU6MtJSY/X6/2p5AIEDXbsmx2WzSaaXqPVTpWanRkDLlG3OS9MlYX+etra3shkZdCxhHwWuSpGUHEgZJuTPArVDxg8EguwmjaLqF6Em9Mo0HKeAbrSB1USZjudSls9/ZrVhs+3IpUKYg5uCrOwCxpKGhITc3Vx2+adMmj8fjcrmkuudyuTwez6ZNmwghTU1Na9assdvtra2toii2traWlJR4PB58kwTWRE/nFIfDoWhpaHh1dbXH43E6nTTQ6/USQtasWZMgowEAkVJXV0crLx3ABINBu93u8Xjo8V0UtlsAICkIhUK5ubk2m432x4LBoMPh8Hg8dXV1JFwrhr4cSAAYugMQMyoqKvLy8lwulzpqzpw5hJApU6ZIIfRvGr59+3ZCyMyZM/v160cI6devH31x29jYmBDDATABQ+f79+8nhAwdOlTzh+vXryeEFBYW0v+OHj2aYOgOQDJAK+/s2bNzcnIIITk5OWVlZYQQ+i9hugUAkoi9e/cSQqZNm0b7Yzk5OcXFxeR0FWC3YujLgQSAoTuIFp/PV1paKi37aWpqkqKampoqKipoFF3zI0VJK5rourj8/HyPx0OjqquraaxmenksYwWRZFV+fr7P5zNosBzNJbvshVhlZWVut1s+PpdQz7CSh7e1tRFCevfuLYX36dOHELJ79269a4EEA51LMHTOxu12i6JIu/6EEPoo0NcHVgYVn0IrrzxEqsiUiN0CsAiQOmXr1q2EkOHDh0shOTk5oijSTe/ZrRj6ciARxHc+PrA20a+i0TzAw+v16kVJM8bpfxVp/H6/3W43nl5aQaQQsyITQojdbg9rsAL9ShP+iamT0esqJsyTn/aHwmZiFqyS0sPsk4HONVEnczgc9AadTieNlWbYqpORn9YIg0DVgA2J3bpoVHwGzc3NRKsKm8rEIARr3Q0QjW+E1CXoBxVRFF0ul3S8vGJPB1GnFdPMlnGtsKC9A2ogiLQmeqdAXRJd1SOKYn19PSGkpKREiqqvr6dR0v5V8h/a7XbarafrhTRD5Oml1Uetra3Up1NPLU9JfyjlQ5fkkdP7iDAMjiGantrr9cq/vdtsNqmZibm7p8Dp62H2yUDnjMciD5F6M3LkG/xIxjscDlodzI7eoWrAhsRumIeKz8But6urtoihOz+i8Y2QuuJRKN4aGGzFNMUfTY1AewfUQBBpTfROQXol6fV61U24KIqBQMDv97vdbskP0nD6t/xFJg2RZ6JO39zcLMXS9kPetNBweiF5PsFgkJzeOiuswTFB01OrRzXSTicxd/cUOH09zD4Z6FwTtURpiNTJE0/PLtEbn9OP85qfSvSAqgEbErthHiq+HvJBlILoWy7NPDF0D0s0vhFSV1hrt9ulmzLeisW8L4f2DqiBINKa6J2C/AgN+Wdkinqyk8J9yxOzQ9gOUZ1Sk7AGqzPXy4eNOpna9ctDYu7uKXD6eph9MtA547dGkumdjkO7YqbOzoGqARsSu2EeKr4mjHG73r1ESQzLNIWJxjdC6or06lcGRlox9t1FANo7oAaCSGti5RT8fr/0Sdlms9EWnb6JLCkp8Xq9fr8/EAhYwd0zDNbM3Li7N34v8kDq7vXeLkvLuiIDTl+PyJ4MdK5nWDTJDGYiAVUDNiTWwzxUfIlAIEDnycu/muqZHUNiXqYpSUw+xkDqtD+mZ6EmUmzM+3Jo74AaCCKtia1TaG1tlS9qUng66r+idPfSuibx9B451CHKU9KjOIxMoFIYHEOM3J08kH6Bl3eG6BtlaUZ9ZMDp6xHNk4HONS2n0EmM6o4LnQypjqW9QFMLFKFqwIbEZ5iHiu/3+202m3wisSZxcjUYuoclVr4xzaVO+2NyC423YjHvy6G9A2pwOByICnoyR0NDAyGkX79+/fv3VyRoaWkhhIRCIc39q8xSWVlJz95oa2urqqoihIwaNUqRpqCggBDicDja29tpiM/nEwShoqLCiMFxgt6+z+eTDkGhx6XQ8Msuu4wQUlVVJd3d66+/TggZNmxYYswDbKBzg0ybNo0QUldXJ4XQv6m1NLa2tpZGhUIhenc0FgCrgYov0dbWlpubO2TIkPLy8l69esUqW2ARIHUJeixcZWWl1Fujrdj48eNJuFYMfTmQCHi/OwA8if59Ht3VUwF9v6h3XDN9H6mWHzuE/q1YcCXNQVL8Vr0uy2az0Q8FDINjiPpeAoGA+mh3ySrx9KtcOdHvlYr3tXqYfTLQuSbqewkGg2oly+cKsmONAFUDNiR2X2hR8SWk4x7VKFJqBkZJDMs0hYnGN0LqctS3LO+PmW3jounLob0DaiCItCYmTkF+gKfdbqcHlVOkxt5utzc3N8snDkXm7sXT+7QrdiVR/9blctHZVvSK8gl+DINjhdoeURSDwaB0TKjNZnO5XIo5V06nU4rVPA3bLHD6ekTwZKBzNWF1TpdHKhKwY8MCVQM2JKbDPFR8uQ2aaKaMyUXleWLoHpYofSOkLqe+vp5el/bWFLGMViy2fTm0d0CNIDI9Mkhtamtri4qKkkIDgiAQ2bIrYIQkKt8EY9knA52HxbJlByyCIAg1NTWFhYW8DTEBKj6bZCzTxJMUvhFSN0VSlClIMFjrDgAAAAAAAAAAWBoM3QEAAAAAAAAAAEuDoTsAAAAAAAAAAGBpsngbAIAhsNQHpAPQOQBpCCo+SBMgdQCiBF/dAQAAAAAAAAAAS4OhO4g7giDQPUWtYIamMR6PRx0YCoUqKytp+kWLFrW0tERwUc2cCSEtLS2LFi2imVdWVra3t6uNZBCBJSDBWKSkNGVTXV2dn58vCEJpaWlTU5M8fSgUkmLz8/Orq6tDoVAE11JfVC8WwgYpg0VkzKhi0TQles2ZRFNTk5QA9Tr1sEhpqqVlXNtyiZqNVRO2udTs6aFqgGjhcCAdsAyJOTHSIkrTkz09oVRtIT2WU47f7zd1Rb2cpXAJm81GT/7Uq6cKYwwagBNB9UjAk7Gs7NXClg6tDQQC6libzSY/SleP1tZWTemGjTUrbKgasCH8zgC3Zq3XrHq0ahvPU685kwgEAvIEZus1G45lmkTE2zdaU97spkeOQqKmYjXTs5tLvZ6eqaqB9g6owVd3kF5Q3Uv/bWhoyM3NVSerrq72eDxOp5Om93q9hJA1a9YYv5BezqFQKDc312aztba2iqIYDAYdDofH46mrqyNaDpp6f4fDobAcAONI4qHCdjgc9FWRKIoul2vq1KltbW2EkE2bNnk8HmkkT2M9Hs+mTZsMXkgSqkTYWAgbgHigqGKazYrBrPSaMzmLFy/WvDoA8UAhMHbTQ1FI1FSsGnZzyejpoWqAKMHQHaQvFRUVeXl5LpdLHbV+/XpCSGFhIf3v6NGjiZmhOyPnvXv3EkKmTZvWr18/QkhOTk5xcbF0RQXt7e25ublOp/PSSy81eGkAGFCZFRcX5+Tk0JBbbrmFELJ582ZCyJw5cwghU6ZMkdLTv2k4m/379xNChg4dGkEsACBhmG1WGM2ZPM3hw4djZCAAJjDYuLAlGoGA2c2lqZ4eAKbA0B2EJxQKCYJQWlqqCC8tLRUEga7taWpqqqiooKt36JofzazUy3vUIT6fj+acn5/v8/n0rIp+/V5ZWZnb7ZZ7Xgm32y2KojS88Xg8hBB238Vgzlu3biWEDB8+XArJyckRRdHtdqsTr1ixwmazzZ492+B1QQxJSdlTJUvClv5ubGwkqnUZEnrhAKQYKVnrFZhtVhjNGcXn85WVlZWXl0dgDEgk6SBvTdgSjUzA7ObSVE8PAHOoZ1KB9MH4Kho6s06+5JWuC6KTlDSdkTSPSK40teoUIXa7XZGP3W7XNMmsqiOLkqYUyqdFGUedM3Xroii6XC76t8Ph0FxLTGfpe71eg6aqwSopPQw+mdSTPZWcNFteFEVp6Z10R4oZgOT0mywjz8rv9zudTpqh0+mULsSO1XxEekDVgA2Jbl106tV6OZrNikE0s21ubpaeQNhbjpgoyzRNMOIbU0/eYRsXtkTZsQzYzWXYnp7Ba6G9A2ogiLTGuFOgS+PUTqq5uVk87YPq6+tplLRrCP2vcXdPexV2u5263WAwSL2/2f3hNDHeEsjxer0Oh4N63ghG73r3q2jVpG3q5NhsNvU2QqaaFjh9PQw+mdSTPbVfuiPpWnJj5B8TbDabwV6+5rpZSdjsWLWdDKBqwIZEN8xLvVovR7NZiTjbYDBos9mkIRn7lqMhyjJNE4z4xtSTN7txYUs0rIDZMJpLGsLo6Rm8Fto7oAaCSGtMOQVFk2+z2UpKSuQJAoGA3+93u92KwYBxd09/qP4kqN6AJAIYjtKID6UvdM1+rNC7X7vdLr1/VYymKPX19UTra6eppgVOXw/jTybFZE97KvLOhMJsdTdI2qzRyIWkPp/4U2GzY9V2MoCqARsS9TAvxWq9hF6zEnG29C6ktox9y9EQfZmmAwZ9Y4rJm924sCUaVsBsGM2l1Lzq9fQMXgvtHVADQaQ1ppwCfY1KX83SKUbycax6clQE7l6dgyIr9Q8NJta8tJEoCdrwmP1eoXe/6iZNkXNJSYkimXFTJeD09TD+ZFJP9oFAgL6HstlsilmC6rdImu+VjMOuMvJY48KGqgEbEvUwL/VqPUWvWTGIIlvqHORDJvYtR0P0ZZoOGPSNqSpvRRrawDEkakTADNjNJc2K0dMzeC20d0ANBJHWmHIK1O/Qd4rUQ0leiY4ESkpKvF6v3+/XPN9V/bdebOLdvUEfasqt6/2EtovsZPQBai4MM2UDnL4exp9MCsueQqdE0o8hmomJ+TdWBq/OfkR6QNWADYl6mJeStZ7RrBhEkS3DqgjqddhLY+geFoO+MSXlbTxPg4R9jJrJyOnmMmxPz+BV0N4BNRBEWmPWKVC3Tr25fBqtwgfJN74Smd5K0TBE+U2AjamWQL2bFzVVMa8sgovSlpIe9Umhj0ueM2Nao0F3T4HT18PUk0kl2auFTb/AMDbpMSg5vQ3wqLDZscavIkLVIBwkFsO8VKr1lChny6uzJUwYlkR2aQzdw2LcN6aSvBmNC1uiRgRs3AxFYNiensGroL0DanA4HDDByJEjCSG9e/cmhIwdO1YR29LSQggJhUKau4ZQqJNtaGigKVesWCGPLSgoIIQ4HI729nYa4vP5BEGoqKiI4V0YYdq0aYSQ2tpa+t9QKFRVVSVZGA30sJDKykp6EAshpK6ujhAyfvx4Kc0nn3xCCBkwYECU1wIxIZVkrxB2S0vLhg0byGlZ0lvw+XySOOnJQIxbU+RMxUyhf9O7Y8cCYDVSqdZTYt6sqDuU8vBYXQXEg1SSN6NxYUs0SgGzm0sjPT0AIiQW43+QrETwPo++yFR8fNY78Fy+Z6lmSqlhkLJSr7Oy2WyaZ6eZhSF4zSj1oZ0RbEOqmVL9uBTPkz5kzbs2VW3xvlYPs08mZWSv3qaOyFbrBQIBdaxkCVt7mjlLVYYdGzZzOVA1YENi9IU2ZWq9/HbU+RuvemFTqhMYzzzspfHVPSymfGPKyDts48L4LTuWnZjdXIrhenoGqwbaO6AGgkhrInAKdHqtfGMPinSipt1ub25upgeQ0IlYCg8lnXKpGUsTSDOdnE5nTHy95oXCRkmm0qVfRrIymHN9fT29R2mrsChN1QROXw+zTyaVZC9tU0fNVpzWEwwGJVOpOI0fZiP/raLKhI01LmyoGrAhMRrmpVKt1wtkhEeQUp3AVIPFzhlD97BEsPFwasib3biwf8uINdXkKZpLCqOnZ7BqoL0DagQR85rSmNra2qKiojTRgCAIhJBY3awg8Kk7pu4ircrXFOnzZJJF9sbtTJ+yA5EhCEJNTU1hYSFvQ7hhttZboV6HzSfNy9QIaeIbY9uohb0W36qRJmUKTIG17gCYpqGhQXozDUCaANkDkHqgXgOgCaoGsCYYuoP0QhAE+rIzGrZu3Tp79uyY2GOcmFgO0hMryx7CBiAeGKxZqNcgGUmAwFA1gDXB0B0A09x33328TQAg0UD2AKQeqNcAaIKqAaxJFm8DAEgQyb5YKNntB1ywvmysbyEAyYUV6pQVbAApSbJLK9ntB9zBV3cAAAAAAAAAAMDSYOgOAAAAAAAAAABYGgzdAQAAAAAAAAAAS4OhOwAAAAAAAAAAYGmwTR0gtbW1Efyqs7MzMzMz5saAGFJfX8/bBEsTmfIBX6BqEBaIBKQtaNcsjqnOM1wZUIOhOyBFRUW8TQCAA1A+ACnJsmXLli1bxtsKADiAdg2A1AYT5tOawsJC0SRdXV3Lli0744wzbrzxxi+++MLsz/lSU1NDCOFtRaLhrTIrEoHy48fEiRMvueSS48eP8zbEHPv27cvJyZk7dy6Xq/NWELAuXAQZlhMnTgwYMGDSpEm8DQnP4sWLe/TosW/fPt6G/ITCwkLeyrI6lmrX4kry9uVCodD06dMFQZg/f77xRp+3soC1wNAdmOCrr776wx/+UFZWtnDhwrfeeqtPnz68LQIgudm8efP//M//PPfcc926deNtizn69+//yiuvrF69uqqqirctAFidZ599tq2tzeFw8DYkPHa7fcCAAcXFxRgzABBbzj777FdffbWmpmbdunX/9V//9cknn/C2CCQfGLoDo3i93iFDhuzZs+e9995bsmRJRgbEA0BUnDhxYv78+UVFRePGjeNtSyTceuut8+bNmzt37p49e3jbAoB1OXz48BNPPLFw4cILLriAty3hycrKeumllz788MM1a9bwtgWAFKSgoGDXrl05OTlXX3318uXL8Y4MmAKjLxCejo6OJUuW3HzzzcOHD9+1a9fw4cN5WwRAKvD0009/8cUXFRUVvA2JHIfDMWTIkMLCwh9//JG3LQBYlLKysl69et1///28DTFKbm7ufffdd//99x84cIC3LQCkIBdccMG77777wAMP3Hfffbfddts333zD2yKQNGDoDsLQ2tp6ww03LF269Nlnn3399dfPPfdc3hYBkAq0trYuXbp08eLFffv25W1L5GRnZ7tcrkAgMHv2bN62AGBFtmzZUlNT89e//rV79+68bTHBkiVLfvvb386ZMwefBAGIB1lZWUuWLPnggw8+/vjjgQMH1tXV8bYIJAcYugMWGzduHDp06Hfffbdt27YFCxbwNgeA1OGee+654IILUqBa/eY3v1m3bl1NTc0LL7zA2xYArEVnZ+c999wzZsyY/Px83raYo1u3bi+++OK77767du1a3rYAkLLk5eU1NjbeeOONf/jDHxYsWHDy5EneFgGrg6E70Ob48eMLFiyYNGnShAkTduzYccUVV/C2CIDUYdOmTW+88caKFSuys7N52xIDbrnlloceemjevHmNjY28bQHAQqxcuXLv3r3PPfccb0Mi4Zprrpk3b96f/vSnQ4cO8bYFgJQlJyfntddeW7t27UsvvTR8+PDm5mbeFgFLg6E70GDv3r1XX331K6+8Ul1dvW7duh49evC2CIDU4dixY3/605+mT58+evRo3rbEjPLy8hEjRhQVFYVCId62AGAJ2tvblyxZct999w0YMIC3LRHyxBNP/OIXvygtLeVtCAApzqxZsz7++ONu3boNHTp0+fLlvM0B1gVDd6CEHlnRvXv3xsbGoqIi3uYAkGo8+eSTX3/99dKlS3kbEksyMjKqqqqOHj06a9YsLI4FgBCycOHCnj17Pvzww7wNiZwePXpUVlb+4x//cLlcvG0BIMW58MIL33vvPexdB9hg6A7+j++//37q1Kl33HFHcXHxli1bLrroIt4WAZBq7N+/3+FwPPbYY7/61a942xJjevfuvX79+jfffHPZsmW8bQGAMzt37ly7dq3D4ejZsydvW6Lihhtu+O///u958+YFAgHetgCQ4tC9695+++0dO3YMGjRo8+bNvC0ClgNDd/D/s3379qFDh/7rX/+qq6tbvnx5aizBBcBqLFiwoH///nfffTdvQ+LCyJEjy8vLH3jggS1btvC2BQBudHV13X333cOHDy8sLORtSwz4y1/+cuaZZ6bAnpoAJAWjRo369NNPb7jhhltuuQV71wEFGLoDIori8uXLr7vuuosuusjv948dO5a3RQCkJn/729/q6upWrlyZwq/GHnzwwQkTJkydOvWrr77ibQsAfHjxxRcbGxtXrVolCAJvW2LA2Wef/dJLL9XW1m7cuJG3LQCkBTk5OS6Xi+5dd+2117a0tPC2CFgFDN3Tnfb29vHjx5eVlT388MObN2/+5S9/ydsiAFKTo0ePlpWVzZo1a+TIkbxtiSOCILz00kvZ2dlTp07t7OzkbQ4Aiea7777785//fM899wwePJi3LTHjxhtvvP322+++++5vv/2Wty0ApAuzZs3auXNnV1dXbm4u9q4DFAzd05p33nknNzf3s88+e//995csWZKRAT0AEC/Ky8u//fbbp556irchcefcc8+tqanZsmXLk08+ydsWABLNokWLBEFYvHgxb0NizLJlyzIzM++9917ehgCQRgwYMGDbtm0PPPDAvffeO2nSJLw7AxiqpSkdHR1LliwZO3bstddeu2vXrry8PN4WAZDK7Nu3769//esTTzzRp08f3rYkgquuusrhcCxZsuStt97ibQsAiePTTz99/vnnn3766XPOOYe3LTEmJydnzZo1r7zyitvt5m0LAGmEtHfdtm3bcnNz33vvPd4WAZ5g6J6OtLa2Xn/99UuXLn322Wc3bNiQej0MAKzG/PnzL7300pKSEt6GJI577rln2rRpM2fOPHz4MG9bAEgEoijec889ubm5t99+O29b4sKECROKiopKS0uDwSBvWwBIL0aPHv3pp58OHz581KhR2LsuncHQPe3429/+lpub+/3332/fvh0bxgKQAGpqajZv3vz8889nZWXxtiWhrF69+rzzzisoKDh16hRvWwCIO6+99toHH3ywatWqFF59tnLlyo6Ojoceeoi3IQCkHeecc051dfXatWtffPHF6667bt++fbwtAhxI2dYFqDl27NiCBQsmT55ss9m2b98+aNAg3hYBkPr88MMP991331133ZWGy1J69uxZW1vb1NRkt9t52wJAfPnhhx8efPDB4uLiYcOG8bYljpx//vnLli1zOp1vv/02b1sASEfo3nUdHR1XXnml0+nkbQ5INBi6pwt79uy5+uqrX3nlldra2nXr1vXo0YO3RQCkBY899tjRo0fTYXc6TQYNGlRZWfnMM8/8/e9/520LAHGkvLz82LFjjz/+OG9D4s7UqVMnTpw4Z86cI0eO8LYFgHTkd7/73bZt2+69997S0tKCgoLvvvuOt0UgcWDonhasW7fuqquuOvPMM3ft2lVQUMDbHADShT179ixfvvypp576xS9+wdsWbkybNu3OO++86667Dhw4wNsWAOLCvn37nnvuuccffzxNavqqVatCoRBm0wDAi+zsbLoR7Icffpibm/v+++/ztggkCAzdU5xQKDRlypQ777yzuLj4/fffv/DCC3lbBEAaQc92Li4u5m0IZ1atWnXhhRfedtttx44d420LALFn/vz5l1xyyZw5c3gbkiD69OlTUVGxYsWKDz74gLctAKQvN954Y1NTU25u7ujRox966CFsK5MOYOieymzbtm3o0KHvvvtuXVS21tgAACAASURBVF3d8uXLs7OzeVsEQBrx2muvvffee6tWrcrMzORtC2e6d+9eW1t78ODBsrIy3rYAEGM2bty4efPmlStXptU+lHfeeefNN99cXFyM93EAcOT888/ftGnTSy+9tHLlyuuuu27//v28LQLxBUP31EQUxeXLl48YMeKSSy5pamq6+eabeVsEQHrxww8/PPDAA7Nnz7766qt522IJ+vfv/8orr6xevbqqqoq3LQDEjGPHjpWVlc2YMWPkyJG8bUk0L7zwQnt7+6OPPsrbEADSnVmzZu3YsePkyZPYuy7lwdA9BQkEAuPGjXvwwQfLy8vr6up69+7N2yIA0o5HHnnk+PHj6bBnlXFuvfXWefPmzZ07d8+ePbxtASA2PP30019//fXTTz/N2xAO9O3b98knn3Q4HDt27OBtCwDpzmWXXdbQ0FBaWlpaWlpYWIi961IVDN1Tjbfffjs3N/fgwYP19fUPPvhgCp8uC4Bl+fTTT1etWrV06dLzzz+fty3WwuFwDBkypLCw8Mcff+RtCwDR0tbW5nA4HnnkkV/96le8beFDSUnJqFGjbr/99hMnTvC2BYB0p1u3bk8//fTmzZu3bt06dOhQbEWRkmBclzqcOnVqyZIl48aNGzNmzEcffTR06FDeFgGQjoiieM899+Tm5t555528bbEc2dnZLpcrEAjMnj2bty0ARMv8+fP79u07b9483oZwQxAEp9P5+eefp+35lwBYjZtuusnv919xxRXYuy4lwdA9RTh48OD111//7LPPrl27dt26dT179uRtEQBpyrp16z744IOVK1dizosmv/nNb9atW1dTU/PCCy/wtgWAyHn77bc3bdr03HPPdevWjbctPLnwwgsfe+yxJ554orGxkbctAABCCPnFL37hdrtXrVq1YsWKESNG/Pvf/+ZtEYgZ6FmmAhs2bBg6dOjJkyc/+uijmTNn8jYHgPTl+++/X7hw4dy5c4cNG8bbFutyyy23PPTQQ/PmzUNfHyQpJ0+enDdv3qRJk8aNG8fbFv4sWLDgmmuu+eMf/4jvewBYBEEQ5syZs2PHjuPHj//+979/9dVXeVsEYgOG7snNsWPHFixYUFRUNGvWrA8//PCSSy7hbREAac3DDz/c2dn52GOP8TbE6pSXl48YMaKoqCgUCvG2BQDTPPvss3ShO29DLEFGRsYLL7zw2Wef4YEAYCkuv/zyhoaGO+64Y9asWYWFhcFgkLdFIFowdE9idu/ePWzYsPXr17vd7uXLl6f5nD0AuLNr1641a9YsXbr03HPP5W2L1cnIyKiqqjp69OisWbNEUeRtDgAmOHz48BNPPLFw4cILLriAty1WYcCAAY888sijjz66e/du3rYAAP6P7t27L1++/J///OeWLVuGDh26detW3haBqMDQPVlZt27dVVdddf755/v9/gkTJvA2B4B0p6ur6+67777mmmtmzZrF25bkoHfv3uvXr3/zzTeXLVvG2xYATFBWVtarV6/777+ftyHW4v7777/iiiv++Mc/dnZ28rYFAPATbr75Zr/fP3DgwBtuuGHJkiWopMkLhu7JRzAYLCwsvOuuux544IF33nmnb9++vC0CAJCXXnpp+/btK1euFASBty1Jw8iRI8vLyx944IEtW7bwtgUAQ2zZsqWmpuavf/1r9+7dedtiLbKysl588cVdu3Y999xzvG0BACjp1auXx+NZtWrVM888c9111x04cIC3RSASMHRPMt57770rrriioaHB5/MtWbIkMzOTt0UApCMOh0N+Yup333338MMPz5s3Lzc3l6NVyciDDz44YcKEqVOnfvXVV/LwL774oq2tjZdVAFDWr1//8ssvS2s6Ojs777nnnjFjxuTn5/M1zJoMHjx44cKFdrt9//79UuDRo0efeOIJjlYBACh077rt27cfPXr097///WuvvcbbImAaDN2Ths7OziVLltx4441XXnnlrl27rr/+et4WJQGHDh3KzMwUTlNUVEQIEWRcd911vG0ESckLL7wwcuTIGTNmBAIBQsjChQszMjIWL17M267kQxCEl156KTs7e+rUqdIUPp/PN2TIkLVr13I1DQCyefPmu+66a9iwYTt37iSErFq1au/evfiqzMButw8YMKC4uJi+73j//fcvv/xyxWAeAFOgLxdbBg4cuG3btttvv33mzJmzZs06cuQIb4uAGURgJQ4dOrRhwwZ1+Oeff3799dd369Zt2bJlXV1diTcsebnuuuv0JjALgrBy5UreBoLk48iRI/TM9uzs7DPPPPP+++/PzMx89dVXeduVxGzfvr1bt26PPfZYZ2dneXl5RkaGIAi/+93veNsF0p3+/fsTQrKysgRBmDVrVk5OzkMPPcTbKKvT2NiYnZ397LPPzp07VxCErKyszMzMyspK3naBJAZ9uXjwz3/+85e//OUFF1ywdetWdeyxY8c2bdqUeKsAG3x1txCiKN5xxx233377vn375OFutzs3NzcQCDQ0NCxYsAAraU0xc+ZMOspSk5GRUVBQkGB7QAqwc+fOrq4uQsipU6d+/PHHioqKnj17/va3v+VtVxJz1VVXPfPMM48++ujVV1/9yCOP0BeUn332WXNzM2/TQPpy9OhRuhy0o6NDFMXq6uqTJ0+ee+652OGJzdChQ6dPn/744487nU5RFDs6OgRBePfdd3nbBZIY9OXiwdixY5uami6//PKRI0eq9667//77i4qKPvvsM17mAU0wdLcQa9as8Xq9J06cmDx58okTJwghJ06cWLBgwcSJE8ePH79z504so42AgoICzZcdGRkZo0eP7tWrV+JNAsnOjh07srOzpf92dXUdOXJkxIgRM2bMaG9v52hYUpOXl9ezZ8+mpibx9Lri7Ozsv//973ytAulMY2MjfUlHOXny5LFjxxYuXHjFFVfIt7oAckKh0OzZs1955ZVgMNjR0UEDOzo63n77bb6GgaQGfbk40atXrzfeeMPhcDz99NM33XTToUOHaPibb765atWqkydPFhYWnjx5kq+RQA6G7lbhwIED9913nyiKnZ2de/bseeihh5qbm/Py8l5++eV169atW7euZ8+evG1MSs4999wxY8Zo7uc3c+bMxNsDUoAdO3bIO/SEEPquuqam5uKLL169erWIg8pN4nQ68/Lyjhw5curUKSmwo6Ojurqao1UgzaETvxWBXV1d+/bto1tdhEIhLoZZFrfbfemll77yyiuiKCqcZHt7+7///W9ehoFkB325+CEIwoIFC3bu3PnNN98MGjTI5XK1t7fPmjVLEISurq69e/c++uijvG0E/weG7pagq6trxowZ8vfTy5cvHz16dGZmZmNj44wZM/ial+zMmDFD0YcghGRlZWGLYBAZW7du1Zwx29XVdfz48ezsbKxqMU4oFLrttttKS0s7OjoUT1UUxY8//ri1tZWXbSDN2blzp+ZrODp/vn///meffXbirbIsDQ0NkydP/uqrr+Qv4CQyMzMxZx5EA/pycWXQoEENDQ1FRUXTpk275pprvv/+e/q0Ozo6nnrqKVRe64ChuyX4y1/+sm3bNkVr9/3331dXV9M9ckA0TJw4sVu3bvIQ6utzcnJ4mQSSl6+++urw4cPq8Ozs7LPOOuutt94qLi5OvFXJS11dXV1dnd7LjszMzI0bNybYJAAoH374ofRKXSIzMzM7O7uqqmrJkiV4SSfnmmuuqa+v79u3r3qqAiEEy91BlKAvF2969Ojx/PPP33XXXQcPHpQPSQRBmDp16nfffcfRNiCBoTt/du/evXjxYsWrRFEUT5w4MWvWLGyHEz09evS49dZb5Z2Jzs7O6dOnczQJJC87duxQB2ZnZ1900UW7du0aNWpU4k1KaqZMmbJ79+5rr71Wcwuizs5OzJkHXDhy5Mj//u//KgLpGzqv14vZcJpceeWVfr9/1KhR6uqM5e4gStCXSwC7d++mx+XIA7u6ur755pv58+fzsgrIwdCdMydOnCgoKNCcknfq1Klt27Y9+eSTibcq9Zg+fbr8DeLPfvazcePGcbQHJC/bt28/44wz5CGZmZmjRo3atm3bhRdeyMuqpOaiiy569913V69e/bOf/UzxvU4UxR07dnzxxRe8bANpy65duxSv1LOysi6++GK/3z9ixAheVlmf884775///OeTTz6ZkZGhWJkcCATojv0ARAb6cnHl+PHjBQUF6lUJhJBTp069+uqreJNuBTB058ySJUtaWlrUU/IonZ2dS5YswU620TNu3DhpUWJ2dnZRUVH37t35mgSSlIaGBsXalrlz59bV1WHOXjQIgjBnzpw9e/aoP79nZmZin3mQeD766CP5iyTpDR2OgQyLIAgPPvig1+s955xzFM8Qc+ZBNKAvF1fuvffevXv36g1JaDMtbUEPeIGhO08+/PDDpUuXak6Jp5/1evToMWnSpGPHjiXctFQjOzu7sLCQ9iFOnTo1bdo03haBZGX79u10mkxGRkZ2dvbatWufe+45vfNmgSkuuOACn8/38ssvn3XWWVKPv6urC2/6QeJR7FFH39BhXzrj3HDDDU1NTVdeeaX07R3L3UGUoC8XP3744YdQKERdnGJqIUUUxePHj0+fPl3zszxIGAIOMeLFjz/+OGjQoEOHDsnfb2VlZXV0dPTp08dms02YMGHs2LGa9QdEwL/+9a/Ro0cTQs4777xAIKB5xAgAbA4ePEhnxWdnZ+fk5LzxxhtXX301b6NSkP/85z+zZ89+8803CSGiKGZkZPznP//Byb0gkVx88cUHDhzIyMgQBOG5556bO3cub4uSko6ODrvdvnTpUnrQVO/evb/88kveRoEkBn25uNLV1bVr1y6Px7Nx48ZPP/2UfpaQf2LMyMh45pln7r33Xn42pjsYunOjpKTE6XTSXilt0gYPHlxQUGCz2QYPHszbuhSkq6urT58+7e3tCxYsWLZsGW9zQFKyYcOGwsLCjIyMQYMG/eMf//j1r3/N26JU5uWXX54/f/7x48c7OjqcTufs2bN5WwTShSNHjuTk5IiieNZZZ23atOmGG27gbVFys3HjxlmzZh09elQUxQMHDmBbEBAx6MsljEOHDr355ptut9vr9R4/frxbt24nTpwghGRnZ+/cuRNDFV78ZOh+6NChDz/8kKM16YPf73/qqacIIVlZWYMGDRo2bNjvf//7c889N8psCwsLo8whtTVQVVX1xhtvPP7445dccglvW+IFNBBXXnvtNbfbffXVV999992KU2qsQypp4LvvvluzZo3f77/iiivsdjtvc5KG6DVACKmvr//888+jzycZ+eyzzxYvXty7d++FCxf26dOHlxm/+c1v8vLyoszEIuX45ZdfPvPMM4cOHSotLcWrEAXDhw+P/kWwdfx2vEmHvpwc7v781KlTe/bsaWxs3LFjxzfffEMI6du371/+8hfNYyBBzFH6B1FGTU0NP8NADBCjBhpIdqABAA2A6DUgiuLkyZN530e6M3nyZJRjOlBTUxN9QcNvpyrRa0OEH0hmFP4hS51CxBT6OHPkyJEePXrEdl+r2traoqKiWOWWwhqoqqqaOXMmbyviAjSQADZv3jx27FjeVuiSwhr48ssvf/zxx4svvpi3IVYnthqYPHnyhg0bYpVbEvHOO++MHDmS7zelgoKCWGVlqXJ85513brrpJt5WWAhBEGKYm6X8dvxI4b6cHCv78x9++OGss86KVW5AD7V/0Bi6g3jTs2dP3iakL+ng60H8sPK4PbX55S9/ydsEkEZgbBk/8GxB9KAvxx2M23mBA40AAAAAAAAAAABLg6E7AAAAAAAAAABgaTB0BwAAAAAAAAAALA2G7gAAAAAAAAAAgKXB0N0ooVCosrIyPz9fEIT8/Pzq6upQKMRIXF1dLSWurKxsb2/Xi1Vkxf4t4IgpDchpamqSbxEp6BM2FvDFlAbCFqJU00tLS5uamhQXgh+wJmb9AKOUDVZzj8eD6m81zHYJIms7QLyJYXVmd+2AdYhVX46E8+Fox5ORaIZ78sRx7Myrz4SMyfmBejgcjnhfIk6UlJQoHp3NZtNMGQwGbTabOnEgEBBFMRAIMGLZv2UQq7KDBhgY14CcQCCgqGt6lZHmxo5lAA0kAOMaaG1t1SxHKYG6prtcLhoFP2BlTPkBRimHVQjF7/drhusRw7KbPHlyTM4VZ5AmMois7YjV80c5MohVdWZ37diQmJ7rHn0+BkmTQpdQ9+XYPjzidlxOcvlzOekgD3at19SGQbHJUfuHRA/dFbpPFmjnyW63t7a2iqLY2tpKi7a5uVmd2OVyEUKcTmcwGBRFMRgM2u12QojD4RBF0el0yt29PH3Y3zJIoi57OmhAjuQFwmaulxU7VgIaiDemNEAbdb2aS2u6w+GgNV0KoTnDD1iWCNoCvVJmK4RSX1+v6A6GJbm6eukgg4jbjiQauqdDObKrM7trx0bdNY+MBA/d06HQ5aj7ckZa+QjacTnJ5c/lpIM8zNZ6g515BRi6RwgtHvnjpgUgLzAJ+g5GHhIMBqUb13wCUiD7twzQZY83pjQg4XA4pHdyemnoq1y9qs6OlQMNxBtTGvB6vYQQr9ermRVVhdQFFE/XdFrQ8AOWJYK2QK+U2QoRT3+1oF0BDN0thSkZRNZ2iBi6x58YVmd2144Nhu6JJIZ9OSOtvDzEYDsuJ7n8uZx0kIepWm+8M6/OUOEfIl/r7vP5SktL6eR+n88nhdN5/O3t7RUVFdLUfylKnkb6o62tLT8/f9GiRVImipUDisxpAvq3tK4gPz+/tLRUbmEoFKLrkdTGm12B0NbWRgjp3bu3FNKnTx9CyO7du9WJ3W63+NOZEjk5OdLf6skV8nD2b60GNEB0NCA9n7KysvLycr0ElBUrVthsttmzZ0cQyx1ogDA1oIfH4yE/rd3078bGRgI/kCoaYJdyWMrKytxu95QpU4wk5g5kQHRkEEO/kQBQjiSi6szu2lkcFDqJRV9OQXK14wwgD6IjD1O1Ppadefk43vjbHTrrQ47dbpe/b1DYTV9XqC8t/ZbIXmmEzdztdstj6bIB+vbL7/dLRtKphrTyqN9hGHkgivRGAjVpbm6WbpAar55foWmn4rcMEv+1DRrQC6TIC46RjP3WNux3OTnQgGglDdBPpn6/n77EJbKJc6L+BxwjcmIADYhW0gC7lNkKMZK/Jly+0kAGeoFmE8tJ/Fd3lKNeoBiuOpvt2imuyPGrOwpdL5DC6MsZ9+HqrAzC/as75KEXKJqp9aY68+qrx2DCPLXAbrcr1m/Q5yg9fRpLE0uL8hX3Ly9pRebSMgOaOb1hqfA0Y202m1Tq4ulKZXZRgSamylKN3W632WxSffZ6vXKt22w2RnEqfqtHgrvs0AAjUDy9PYnCB2lma7PZ2JviGN/QAhqwlAbodRVIdZn6d/m+dFIbpnlp+IFk1AC7lNkKMZK/Jonv6kEGjECzieUkeOiOcmQEigactqmuneKKvIbuKHRGoBiuL2fch1MMtuNy+A7dIQ9GoHQXRmq9qc68+uoxGLrTx6d+9Ui3XqDWy3dQlN+zZlnKE9P9AGhRUeg+ECUlJVJ6efHIY6kOpN/KNRQlZstSjlzoFHVt11v8oP6tHgnuskMDjEDx9PORbkovGeNNYdhYNdCApTRAw+vr66UQec9PvfcsY+gOP0BJOg2wS5mtECP5a5L4rh5kwAg0m1hOgofuKEdGoGjAaRvv2qmvyGvojkJnBIrh+nL0v0Z8uGimHZfDd+gOeTACKUZqvdnOvPrqMRi6E31ErTsMW5Z6idWBYZ+pzWajkqL7Cuj5TfYt6KU3EqhAXVfVFVuvqpuq5wnuskMDjEBaoHJvrpct9Vx6r2DZsWqgAetoQA8ia2ACgQCdZWez2RgLK+AHNAOTRQMGS1melboLYkpjie/qQQaMQLOJ5SR46I5yZARSGNXZeNdO0wxeQ3cUOiPQeF9OkZXah0c2bhd5D90hD0agaLjWm+3Mq6+e4kN3uvAgEAjQx6dXT8yWpd6bJ/mEDQWBQIDOjVFM4dC7BXlV1/stA3TZraMBRs7yzOluk3oSYsdqAg1YRwN6MDJXHzMDP5B6Ggh7IFzY+woLhu5Wk0HEmsHQ3VLlqEZenfVuwcjHQIKhu9ZPuBc6I2fGc1bERtCOy8HQXe8n3OXBMFIxoNP7uUFITHaYZ7w/iCA3zczpFn+UlpYWKZyijpWmLV1//fWEEJ/Pt379ekLIkCFDNK/CfkZqBg4cSAihBUA5ePAgIaRfv36a6ZuamoqLiwkhL7zwwqWXXhrupgk5vXlpZL9NPNAACaeBsBw4cIAQMmzYsAhirQA0QJgaoJum0g1RKfRvegvq2P379xNC+vbtS/8LP5CSGpCXMlshSQRkQJgyiHnbESdQjiSK6qyH1LWzJih0EkV9DOvDk6IdZwB5kIjkIa/18ejMRzJ0LygoIIQ4HI729nYa4vP5BEGoqKgwmINc6JqZV1ZW0gJra2urqqoihIwfP15Ko44dNWoUjcrJyXG5XFOnTvV4POqtCyPmsssuI4RUVVVJ13399deJTmG0tbXl5uYOGTKkvLy8V69eili6LsLn80kPgR6HQMPZv7UO0ABDA3oOQuEsPvnkE0LIgAEDNC/HjrUC0ADbD0ybNo0QUldXJ4XQv+mt0dja2loa1dLSsmHDBkLI8OHDCfyALPMU0IBeKbMVkkRABmwZmErMEZRjNNWZ3bWzLCj0aPpybB+eLO04A8iD7ROM1Pq4dObloozmQCCbzUa3H1BnKw+RNvmQ70MQNnPF7o6KBIp5CNJxHfIFKtGjPqaP3oL6HqVTIjSfdiAQUGclPT32bxlY4VAoaEDv0WlG0TeL8k07jMdqAg3If85dA+o9jeRGasZK66PgB/QyTwENyHeoZihEjpGil7DI4XCQgfxG2In1sMLhcChHYqw6s7t2bIjFDodDoes9RkUU24dH3I7LseDhcJCHdCNGan0EnXkFJCZr3Skul0ua1eB0OhmbactD/H4//RVdCaAnYpfLRR+HtBeIIiv6SkNvF36aLJonpYZuTyJZpTi8UX4jRB+aIBgMKm5Qyirsb/VIfJddhAb0NaBpjJFAg7GaQAMKuGtAXtNLSkoURko7HhFC7Ha7fKUW/IA886TWAKOUxXAK0cuTDa+uHmQQVgZ6ifVI/NBdRDnGqDorunZsCNehu4hCj6Ivx/DhRB/jpnIfuouQh+FunmatN1viatT+IfKhOxeMPILm5mZi7JV2KsGly84FaEAPaEAONGCFfOIHNKCHFbp6CSO1ZcBl6M6F1C7HsKi75pFhfb8tJ80L3SBp5c/lQB4Sav8QyVp3i0OXQyTdTj8ghkADABoA0AAgkEGqgHJMQ1DogEHayiOLtwGxRBAE+ofdbtfbbBCkNtAAgAYANAAIZJAqoBzTEBQ6YJDm8kipr+50sYHD4SgvL+dtC+ADNACgAQANAAIZpAooxzQEhQ4YpLk8kuyru8jc+MHtdifMEsALaABAAwAaAAQySBVQjmkICh0wgDwYpNRXdwAAAAAAAAAAIPVIhaG7IAjSsgcr4PF41PaEQqHKysr8/HxBEPLz86urq0OhkDy2urpaiq2srGxvb0+s1amGFVQh6EMTKMpdoQoQQ5JCDyBKrPYwNdsCOU1NTeoELS0tixYtoveCtiACrCMDyb2XlpY2NTXxNif5sE5RUvR6d+i/xQqrlTjR8dLsqg0fHiuspgfG+I6aumjRopaWFr2fa2opEuTbzSfXwRIS6hvhiN/v17RHvQUiPepQFMVgMEiXbShiTR1UmD6HQhnECqrQq3S06AOBQPTlLgcaYGB9PcSENNeAFUpZQq8tkAgEAuoE0q/k8jB4OjQlbQ8TkrCIDNTuXX5qsRHS53A4PSxSlBTNGh2T/htJy8PhNLFUiYs6XppdtaP34XLS3J9bSg96bbpaD36/X/1zTS0ZQe0fUuGru3VoaGjIzc1Vhzc1Na1Zs8Zut7e2toqi2NraWlJS4vF46LuZuro6j8fjdDpp3Q4Gg3a73ePx0GMPQPKiV/MdDgchZNOmTR6PR+7xXS6Xx+PZtGkTb8NBXGDrAaQSem2BnMWLFytCQqFQbm6uzWajLUUwGHQ4HB6Pp66uLm6WgrhQXV3t8XgcDofUZXe5XFOnTm1ra+NtGogEvRqN/ltqo/bS7KoNH56q6HkAqgen00nF4PV6CSFr1qxRp1RrKWIwdI8ZFRUVeXl5LpdLHbV9+3ZCyMyZM/v160cI6devH/0I39jYSAhZv349IWT27Nk5OTmEkJycnLKyMkII/RekDO3t7bm5uU6n89JLLyWEzJkzhxAyZcoUKQH9m4aDlEehB5AyMNoCeZrDhw8rAvfu3UsImTZtGm0pcnJyiouLyek2AiQRtMiKi4tps04IueWWWwghmzdv5mkWiAhGjUb/LYXR9NLsqg0fnpKE9QCFhYX0v6NHjyZaQ3dNLUVMtEN3n89XWloqTfGXL/loamqqqKigUXQdrxQlrV6gywby8/M9Hg+Nqq6uprGa6eWxjFXBklX5+fk+n8+gwXIiWJVaVlbmdrvlIzEJ+jaud+/eUkifPn0IIbt37yaEuN1u8aeTaSWPkKRAFZqsWLHCZrPNnj2b/lc9x4YdnrxAD5oo9JDsoJQlGG2BdOmysjL1qTZbt24lhAwfPlwKycnJEUUxiXbThQwo1H55U07/pu/rkwIUpQSjRqdS/w0lrshc00uzq3YK+HAJ6EEirAeQ9EBvVjHI19NS5Mhnb5pdU6GpRa/XqxclzQ2Wblge6/f77Xa78fTSAlHFjSgyIYTY7fawBivQf2Dhn486meYPGbk1NzcTk+virLPGFarQhM6ikWdLr6uYME9OO4IIsI4G5EAPmqj1EBN4aQClrIlmMrl7VySgr+1EUXS5XNK5tWY3v+C4NhIykKDFJ1/gGgwGGek14bjWHUWpiZFkEfTfiAXWuqPE5YT1IVnT+wAAFEhJREFU0npVOyY+XA4vfw49aMJIJq18VNR9hpYMQlT+IaqhOzWCrugQRbG+vp4QUlJSIkXV19fTqNbWVrnF0hOn6qe9WM0QeXpp9UhraystP1oq8pT0h1I+dN0ROb1nAMPgGKIuG83SYhSh3W7ntTVR9PlAFZrYbDb1bmRer1f+jd1ms0UzlrOOBuRAD5po6iF6eGkApcx4LPIQuqmV1NtQJJDunchIom3qIAMJ+h5W6sNJ1zVVNByH7ihKxmNhp4mg/0YsMHRHiUuwvTS7akv3TmQk4zZ10APjsWhGeb1eh8NBu/RyeTC0ZPyisRy6S6+UvF6vpi4DgYDf73e73YpGi/4tfxFFQ+SZqNM3NzdLsVQrchnRcHoh9fswh8NhxOCYoC4bzdLSK0K5Fo1jnWEbVKGGOhH153T1/mTSXhcRYB0NyIEe1OjpIXp4aQClrInayVOrpPtVJKD/tdvtUgJFN9EIHIfukIH8KoqlT8k1dEdRahK2BCPrvxELDN1R4hJsL82u2lJIND5cDi9/Dj1oYsSHO51OcvrVA1tLxi8ay6G7/AgE9QdD9cQGRVEpLGOEaN6toqrIAzUJa7A6c7182Bi5O73AyPy+aKVhG1Shhu5KqPAmaoduERcf26E79KBGUw8xgZcGUMqM30r/pRVc+lihTkD/q+6amJqgwXHoDhnICQQCtA9ns9kimy3JceiOomT8Vi824v4bscDQHSVOCeulRWbVpn9E6cPl8PLn0APjt+w0Uokb0ZIRSGyH7hS/3y99PLTZbNRtUVmXlJR4vV6/3684zo5X0TIM1sw8VkWr965ImkQhimIgEKDzrOQvn4xjtWEbVCFBb1Ne1mz7ubv4eJwNCz1I6OkhJvDVAEpZz7CwWdFktKVgZxIW7ucAQwaa0E9J9AORQbif646i1DNMQZT9N2KBoTsFJc5IrPds5VU7Jj5cDl9/Dj3oGWYkGZuwmchzi/3QndLa2ipfwKCwTLFBC7sg1SH0b2kNg3h63T/t/spTGv+cpTA4hqjzpK9e5D6dvh+SZkf7/X66HCLi3SysOWyDKkT92dGaV4nm0tbUgBzoQYznbHnRGhpAKWtaLoZrzsXTLYX87ujjMrVgj/vQnZLmMlDvZUUzNzWpivvQnZLmRalnuUT0/TdimaE7JZ1LnDARw1XtmPhwOVbw5+msBwXqPNV6oO8yqLVsLRm/qMI/RHU4HN2Fv6GhgRDSr1+//v37KxK0tLQQQkKhkHpNbwRUVlbSU9ba2tqqqqoIIaNGjVKkKSgoIIQ4HI729nYa4vP5BEGoqKgwYnCcuOyyywghVVVVkv2vv/46IWTYsGH0v7m5uUOGDCkvL+/Vq1diTIofUIWCTz75hBAyYMAARTi9fZ/PJx2DQQ/MiMljsQ7QgwI9PSQ1KGUjsPsB5PSRQpWVlZJPqKurI4SMHz8+AeZFD2QgMW3aNEJIbW0t/W9LS8uGDRvIT0+NsjIoSoOkTP8NJU4J66XZVTvZfbgE9GAQhR5CoRC1v6CgIKyWIkeeqdm3O/TzkQL6JVnz5Hpy+suz+tLsEPq3YnGF3pZ96jUYNpuNvg1lGBxD1Pcinn4xI0d6CUcnn2hi/KJW+NpGgSoU0Hdv6vfxgUBArQrJqgiwjgbkQA8K9PQQE3hpAKWsifpewiZQPy6zn2s4fqWBDCTUe1kR8/uYcPzqjqLURH0vMem/EQt8dUeJ66EwKWzVjt6Hy+Hlz6EHTdT3ImqN7/RWRGr+3MhFYzxhXn5Yn91ul88ClTya3W5vbm6WTxGPrGjF0ztyK3YgUP/W5XJJcxWcTqe8l8wwOFZolg3d1oIWsM1mczqd0vwKtdokjF/UUsM2qIJ9FxLBYFA6/JPuaRHN1mWW0oAc6IF9FzGEowZQymrClrVmgvr6emqztAeSKbivjYQMKNJeVjTnCHYv4zthHkWpRu9eNDGVLfehu4gS10FtUtiqHaUPl8N321HoQYFe1ZZ68nQLALM/D3tRhX8QRJnrqa2tLSoqEpnOiAuCIBDZEgugJlZlZ1kNqIEqFKShBuRADyQNNIBSDksMy45OUKTTQS1FmsggVs/fsuVI0qYo2QiCUFNTU1hYGGU+lvXbclDipkh5fw49hEXtH6Ja6w4AAAAAAAAAAIB4g6E7AAAAAAAAAABgaTB0BwAAAAAAAAAALE0WbwMMgVUQQA1UAeRAD+kAShkQyCCFQFGmGyhxIAd6iAB8dQcAAAAAAAAAACwN56G7IAh0d0ErmCE3RtDCeIbV1dX5+fmCIJSWljY1NWleiEGMb8/aWOSW1c8/FPr/2jt33yaaLoyP9b016V6EBJQgaAgNIjRINFwk0xEuEh1BoSalEUi0TkWDnDIS4daQLSKKuEOxKJAjhFAsmkRQOJXNP7BfccS8y8zs2Znd9e54/PwKRPZydnaeZ87MrPcyljrevHnz9evX4/HYNZRR2fF4vLa2pkeeWQ8oeFIDRjkGg8GTJ09o4dra2uHhoWvYKIqSAWdWdE9O2Vj/aQlcaGnByQP22X53d7dh6oxynaK/eHJS9inaEqN/AtZRx5PTVOrc2ADti5q24+wo68k58hWeTJ6Wa5mEb48SmTFbqIbx5IyM1ctIXEQX6ilolydPngwGA2WDSfUFyS/Flf4950z0MtSCUiH7+/uZdcVAH/dLIr/rmKaCskuOU/D2m96Z+OmB4XCo69hsNpPfkLQJpexOG8iPUiqrZtYDCn5aIo5j+nipItxoNLKPKSMwRynCFHnAW5WZBD4ajXKnBftuZTgcJle5OqTe77o74a0H0lK0DWn+ydHS6/2uexH8VNbYAC3FZdpvDmX1cvrwXfdM/JQ1iZI8bdYyCd8ePTJjNifDIJ/nK0ayMIzEpU/3+v0+szZfXyC0/IAb5v9DUb3dbjOGSOP169dRFLXbbTmg39jYuHv37sHBgTCJREN5eawJnRqwRKrw4cOHKIqSGXxjYyOKog8fPlgGMaoshNjd3X358mWr1drf34/jeH9/f3l5OYqiwWAAD3iIFGU8Hs/PzzebTRJuNBq12+0oira2tixD9Xq9+fn5tPigRqQKfALf2tqKoqjT6dDa0WjUarWiKFpfX7c8kE238vTpU2PZwESR9cyk6MwgjH+gY13Imue7ZhuM7RfK1oKx2pXkmbmWT/j26JEZs8EwFWDZpxNFpnudTod22d7eFkK8fPkyuXZCfQGm7gZ+/PghhDh//nyOfV+9eiWEePDgwdzcHC25fv26EOLjx4/6xoeHh/Pz851O59SpU/mLCybAw4cPhRB37tyRS+j/tNwJReXPnz8LIe7fv3/y5EkhxMmTJ+kXni9fvpRUdjARvn//LoS4d+8eCTc3N/fgwQPxp8lnsrq6urCwsLGxMdFCgoLwCZzWLi0t0dq5ubmVlRUhBP3LY9mtrK6u/vr1q8gpgIIUSdFOAwBQL04DsCLDQlANfPI0ri2lwdokbYz264KXuPh0b3Fxkf68cuWKSEzdJ9oXFJ26j8djuolfWf7o0aNGo0GPh+3u7q6urtJt/fTYmDGUft+/vqTb7VLkmzdvdrvdtFLV+GBJFEVCCCmV/L+x13/x4kWz2VxaWpp0qSZKkB7Qb3ThlzMoKtOlvqNHj8oNjh07JoT49u2ba2RvCdISnz59EkJcunRJLpmbm4vjeHNz02b3lZWVzc3N5MWgaSdIlfkEvrm5qVwsT25ZnG63u7Ky8vz58xJjTpQgPVAkRTsNAHwmSGUVwhiAORGwrHzyTFtbvMFaJu2pMFuQ9phcTqbxgIxMB5I/z0y2L0jeIZDvmQq61yj5sB899UF3IBjHtcqz3/r/jUtarZYSp9VqGYtkeb5pB6Iz6vf7nU6HVsk7JDOhqV1y49FoZDw03Vyxvb3NlMSeep9xDc8DVGblhnnxp63ao6tsLANfCZZ49ZxzeJagph3H8cbGBv2/3W7bPOTMh2UW5qBiD4Sqsk0CJ/b29oTds5GZ3UoyVGaFMFT8bGR4HjBuaVn/mf6x1zGu+1n38JRNYhyAZdYG036dlNXLWdmz7kHKyidPZq1rwlfgjyspMtpHPi9oD17iItM9iXziJjkSKLEvEFp+KGHqTs9v6JOcvb09WbidnR1aJV8JoBedV5qs32q1kg8Zir9fCZAb5dDGB58s30dF5558rZE0qLJls9nUX45ir6VCvdO28DxAh0v+xt5sNu27eYmuslHifO1Zwaupe3iWoD+V/sb1NXV6WGZhDir2QHgq2ydwotVqWXqA71boBXhy7MJXCE/FQ73wPFCkhWb6x6ml1zt1D0/ZJMYBGEPmsNBJWb2clU3dw5OVT578WteEnyQzaUuKjPaRz11xkrjIdE+yvb3dbrdpviAPVGJfoOeHEqbusebLZrO5vLyc3GA4HPb7/c3NTabovNK0o34BQ3+7QA6UQyvujDUNGPRXEBsTwc7OjjD9hGuvpULt07bAPBCbmrR8HYUlRpWNEudrzwq1e0AhMEvI5iwvSNunBSYsszAH1XsgMJUtE3iyYJajDQqS1q1QKGktvkJ4qn8jcWAeKNJCM/3j1NJrf8N8YMpK0gZgmaGYYaGTsnrwKt8wH5isfPLk1zolfIXMpE0UHO0jn7viJHFmu3aCfrqnH/lK7Av0/FDO1J2uoNBVGbqBJPn7pH5fRA6l9QhKKH1Hy42NhzZuY3mNdjgckn7NZjPtRhp67Y1+XcdeS4Xap22BeUBvvTnas1FlYxn4SrCkdg8oBGYJ+lPvbJx+ujGeUdrCHFTvgcBUju0SeOw4b09DHkX8PXrgK4Sn+qFeYB4wbmlf/7x/7OPEHkzdA1NWkjYAc0Uk8r+TsnqcKqfuIcnKJ0+b1GqZ8BVsIhMFR/vI50XsQbhKLNzHdYQyJiyrLxATmrpTcek3STK0tCmVe3l5eXt7u9/vG79Ym3YaNSqdbxsjdEtJ8mISVYLxuY7cR6l92haYB4xbCpf2nKZy2uVGm9uueGr3gEJgliDh+G1sSLOWV9rZxwlMZR1jAqf75GlwUwSmtEqx7R1S/VAvMA9kpmgnFP/Y6xh7MHUPTFmCGYC5wp+mU5wqp+4hycrsm4kxuJ7wmTJkRi4+2kc+d1Uws2IzJbaUxnXf3H2BmNDUPf6jKAmZvK9YKRzzmL7R8XJJWVdJjSiHTnu7gHLTiBF9X7qClfy1lrlZK7djfJi2heQBoxBO6qSpnHxwiKCHi2SNTbUHFEKyBAlHn3pOFtsmLTBhmYU5qMUDIamcmcD7/T493+j6hkKmWxEsxnIyVD/Ui8PyQGaKZsj0j1NLr33qHoelLJHjbvnYYljopKxeziqn7nFAsjKZM5PYbsTOlIGJTBQf7SOfu6Icmpe43OkenSbtW2JfILT8UNp33S9fviz+fE/l6tWrytrBYCCEGI/HxlcCEHSevV6Ptnzx4kVy7a1bt4QQ7Xb78PCQlnS73Uajsbq6WtYpSO7duyeE2Nrakkvo/1QGm33fvn1Lfw4Gg3fv3om/vyn19etXIcTp06fLLLQHhOQBKmS326XvYQgh6BsYTOEV0lQ+c+aMEGJ9fZ0+QXRwcPD+/XshxIULF8opuk+EZAlqwmtra9ISlBZu3LhR+rGmi5BU5hP4wcHB/Pz8uXPnnj9//u+//+aIbOxWjF21KGNsWhkheaBIirYZAEwXISlL5BuAFRkWekgwsvLJMzO15m6wlkl7Skf7wdhDZElc4nRvPB6vr6/LfSfbFySdV/DqDl1HUa5VyG/cKSRfV2jcUnpChtIfsWg2mzk+zqSjHEh/u4Cwvp/ZuK9yAY8qylhyPjiDJ7+4BuOB4XCo6yiPZSMTo7IeOVlj0+4BhWAsYSx2jp9cjFvmFl2hLg8EozKfwOX3Y3T0UAp8t8KXKjN4klp+pYkD8kDMpugcKicHAPY6xn786h6HpWyc3jXnUDbZfp2U1Q9d8a/ucXCy5lvLN1gnTZ3MZh8c+dwVJ4mLTPdiU08h9y2xLxCTu2E+/nMzQPK1DYQc7rRarb29veS9Z0rR5QeTjWtpA3l7YafTKUVm44FGo5EsDD3XwWysIN9MQKesv8eIiWCvpYIn07ZQPUAvmXD6DAyzDTlERi7r87CeeEAhJEvEcbyzs0PHku8dYTa2D5tbdIW6PBCSykwCF+kYQykw3UpmqewdUtdQLzwPGFN0phD8AMCppXsydQ9J2bSFzHIJ336dlNUPXf3UPTBZc6/NTPhFSsVEsAyOfO6Kk8RxselenDhNPSeU1Rfo+aERJ4Yjb9++vX37dswOUIKk0WiIxNjLZvsJ1ZJrSSRlaQcP2G8PD4RNlZbILboCPOBKWTUvo9WeFkrUjm78o9v8Asbbll5W/c+IjjreKqvv++bNm8XFxXyHlsxI3i43aWceq96UjnzuSnh9up4fSnvWfXbo9XrMbZNgFoAHgAIsAeCBWQAqhwqUBQqwxIzjrQEwdf+PRqNBV0F4Pn36tLS0VNfRwUSBB4DCpC0B0X2gFBWQFqYatPRQgbJBUkG1I6VPL2H36Zi6O/P48eO6iwBqBh4ACrAEgAdmAagcKlAWKMASM463Bvin7gJ4gQ+PBvlQhlnGh/r3oQxAUo0cEL1e/K9//0s47aClhwqUDZJpr/BpL7/n+F+9xUuIX90BAAAAAAAAAACvwdQdAAAAAAAAAADwGkzdAQAAAAAAAAAAr8HUHQAAAAAAAAAA8BrDa+pu3bpVfTlAQX7+/FliNHhgGoEHADwAyvVAr9eDDeqi1+tdvHixrFDQcUaA0CGBfA50/vfs2TP5x+/fv8fjcX2FAfk5cuTI2bNnFxcXC8aBB6YXeADAA6AsD4iyR43AlePHjy8sLCwsLBSMAx095+zZs9euXTtx4kTBOMjb4YF8DvT80PD/NfoAAAAAAAAAAMAsg2fdAQAAAAAAAAAAr8HUHQAAAAAAAAAA8BpM3QEAAAAAAAAAAK/5P7H2rVpjzW6hAAAAAElFTkSuQmCC", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "from IPython.display import Image, display\n", "\n", "# Vamos a entrenar un árbol de decisión sobre los datos de entrenamiento\n", "from sklearn import tree\n", "clf = tree.DecisionTreeClassifier(criterion='entropy', max_depth=3 , min_samples_leaf=5)\n", "clf = clf.fit(X_train,y_train)\n", "\n", "dot_code=tree.export_graphviz(clf, feature_names=['age','sex','1st_class','2nd_class','3rd_class'])\n", "\n", "tree = graphviz.Source(dot_code)\n", "\n", "tree.render(filename='titanic', format='png')\n", "display(Image(filename='titanic.png'))" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Validación cruzada y selección de modelos\n", "\n", "- Para utilizar mejor el corpus de entrenamiento y no tener que separar datos para validación, una alternativa es realizar validación cruzada. Este método, además, permite disminuir la varianza de los resultados (ya que se obtiene como el promedio de varias evaluaciones), aunque es más costoso en términos de tiempo de entrenamiento.\n", "\n", "\"Drawing\"\n", "\n" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Validación cruzada y selección de modelos\n", "\n", "\n", "- En la validación cruzada, se divide el dataset de entrenamiento en k partes, y se utilizan (k-1) partes para entrenar, y la restante para evaluar el modelo. Este proceso se repite cambiando la parte elegida. \n", "\n", "- Se devuelve el promedio del valor de performance obtenido, y también la desviación estándar de los resultados.\n", "\n", "\"Drawing\"\n", "\n", "Fuente de las imágenes anteriores y lectura recomendada: [Cross-validation: evaluating estimator performance¶](https://scikit-learn.org/stable/modules/cross_validation.html)\n" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Titanic: ajuste de parámetros" ] }, { "cell_type": "code", "execution_count": 40, "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Profundidad 1, Accuracy media: 0.786 (+/-0.010)\n", "Profundidad 2, Accuracy media: 0.833 (+/-0.013)\n", "Profundidad 3, Accuracy media: 0.827 (+/-0.013)\n", "Profundidad 4, Accuracy media: 0.825 (+/-0.013)\n", "Profundidad 5, Accuracy media: 0.823 (+/-0.011)\n", "Profundidad 6, Accuracy media: 0.825 (+/-0.012)\n", "Profundidad 7, Accuracy media: 0.823 (+/-0.012)\n", "Profundidad 8, Accuracy media: 0.824 (+/-0.014)\n", "Profundidad 9, Accuracy media: 0.824 (+/-0.014)\n", "Profundidad 10, Accuracy media: 0.826 (+/-0.014)\n" ] } ], "source": [ "from sklearn import metrics\n", "import scipy.stats\n", "\n", "# Hacemos cross validation para encontrar la mejor profundidad para el árbol\n", "for md in range(10):\n", " clf = tree.DecisionTreeClassifier(criterion='entropy', max_depth=md+1 , min_samples_leaf=5)\n", " kf=sklearn.model_selection.KFold(n_splits=5)\n", " scores=np.zeros(5)\n", " score_index=0\n", " for train_index, test_index in kf.split(X_train):\n", " X_train_cv, X_test_cv= X_train.iloc[train_index], X_train.iloc[test_index]\n", " y_train_cv, y_test_cv= y_train.iloc[train_index], y_train.iloc[test_index]\n", " clf = clf.fit(X_train_cv,y_train_cv)\n", " y_pred=clf.predict(X_test_cv)\n", " scores[score_index]=metrics.accuracy_score(y_test_cv.astype(int), y_pred.astype(int))\n", " score_index += 1\n", " print (\"Profundidad {0:d}, Accuracy media: {1:.3f} (+/-{2:.3f})\".format(md+1, np.mean(scores), scipy.stats.sem(scores)))" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "## Fase 4: Evaluación\n" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "\n", "- El paso final es evaluar la performance del modelo (clasificador) obtenido sobre un conjunto de datos no vistos previamente. Hasta ahora, no hemos dicho cómo medimos esa performance\n", "\n", "- Imaginemos, en principio, un clasificador binario. \n", "- Supongamos que nuestras instancias de entrenamiento tienen una cierta distribución $D$, y que son independientes.\n", "\n", "\"Drawing\"\n", "\n" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Accuracy\n", "\n", "- Lo más sencillo es estimar el acierto (accuracy) o el error.\n", "\n", "\"Drawing\"\n", "\n", "$$ acc_s = \\frac{V_{azul} + V_{amarillo}}{V_{azul}+V_{amarillo} + F_{azul}+F_{amarillo}}$$ \n", "\n", "$$ error_s = \\frac{F_{azul} + F_{amarillo}}{V_{azul}+V_{amarillo} + F_{azul}+F_{amarillo}} = 1 - acc $$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Accuracy\n", "\n", "Ejemplo: tenemos 100 instancias ($n = 100$) donde evaluar una hipótesis $h$, y dos clases posibles {1,0} (consideramos a la clase 1 como la clase positiva).\n", "\n", "Podemos construir la siguiente tabla, llamada _matriz de confusión_:\n", "\n", "| A | h(x)=1 | h(x)=0 | Total |\n", "|-----|----------:|----------:|-------:|\n", "| y=1 | $48_{TP}$ | $12_{FN}$ | 60 |\n", "| y=0 | $5_{FP}$ | $35_{TN}$ | 40 |\n", "| | 53 | 47 | 100 |\n", "\n", "Accuracy (¿cuántos casos predice h correctamente?) \n", "\n", "$$\n", "\\frac{TP + TN}{Total} = \\frac{48+35}{100} = 0.83\n", "$$\n" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Titanic: accuracy" ] }, { "cell_type": "code", "execution_count": 41, "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [], "source": [ "from sklearn import metrics\n", "def measure_performance(X,y,clf, show_accuracy=True, show_classification_report=True, show_confusion_matrix=True):\n", " y_pred=clf.predict(X) \n", " if show_accuracy:\n", " print (\"Accuracy:{0:.3f}\".format(metrics.accuracy_score(y,y_pred)),\"\\n\")\n", "\n", " if show_classification_report:\n", " print(\"Classification report\")\n", " print(metrics.classification_report(y,y_pred),\"\\n\")\n", " \n", " if show_confusion_matrix:\n", " print (\"Confusion matrix\")\n", " print (metrics.confusion_matrix(y,y_pred),\"\\n\")\n", " " ] }, { "cell_type": "code", "execution_count": 42, "metadata": { "slideshow": { "slide_type": "slide" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Accuracy:0.787 \n", "\n" ] } ], "source": [ "# Construimos un clasificador con el mejor parámetro, y entrenamos sobre todo el conjunto de entrenamiento\n", "\n", "clf_dt=tree.DecisionTreeClassifier(criterion='entropy', max_depth=2 ,min_samples_leaf=5)\n", "clf_dt.fit(X_train,y_train)\n", "measure_performance(X_test,y_test,clf_dt, show_classification_report=False, show_confusion_matrix=False)" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Evaluación de hipótesis\n", "\n", "- ¿Es el error en la muestra un buen estimador del error \"real\"? ¿Cuál es la probabilidad de error de este estimador? \n", "\n", "Primero, definamos el error cometido por una cierta hipótesis en una muestra: \n", "\n", "$$\n", "error_s(h) \\equiv \\frac{1}{n}\\Sigma \\delta(y,h(x))\n", "$$\n", "\n", "donde $\\delta(x,y)$ es 1 si los valores de $x$ y $y$ son distintos, y 0 en otro caso.\n", "\n", "El error real es simplemente \n", "\n", "$$\n", "error_D(h) \\equiv P_{x \\in D} (y \\neq h(x))\n", "$$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Evaluación de hipótesis\n", "\n", "Si \n", "- Nuestras hipótesis toman valores discretos\n", "- Nuestras instancias son independientes entre sí, y de la hipótesis\n", "- $n \\geq 30$\n", "- $error_s(h) = r/n$\n", "- $error_s(h)$ no está demasiado cercano a 0 o 1 (Regla: $n\\cdot error_s(h)(1-error_s(h))\\geq 5)$\n", "\n", "Entonces \n", "\n", "1. Un estimador no sesgado de $error_D(h)$ es $error_s(h)$\n", "2. El 95% de las veces, $error_d(h)$ cae en el intervalo (llamado _intervalo de confianza_):\n", "\n", "$$\n", "error_s(h) \\pm 1.96 \\sqrt{\\frac{error_s(h)(1 - error_s(h))}{n}}\n", "$$\n" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Evaluación de hipótesis\n", "\n", "Ejemplo: supongamos que nuestra hipótesis se equivoca en 12 de 40 instancias. Entonces, el intervalo de confianza 95% es: \n", "\n", "$$\n", "0.3 \\pm 1.96 \\sqrt {\\frac{0.3 \\times 0.7)}{40}} = 0.3 \\pm 0.14\n", "$$\n", "\n", "Si evaluamos sobre 4000 instancias, y nos equivocamos 1200 veces, el intervalo es: \n", "\n", "$$\n", "0.3 \\pm 1.96 \\sqrt {\\frac{0.3 \\times 0.7)}{4000}} = 0.3 \\pm 0.0014\n", "$$\n", "\n" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Evaluación de hipótesis\n", "\n", "¿De dónde salen esos valores? \n", "\n", "- Evaluar una hipótesis n veces y ver cuántas se equivoca es como tirar n veces una moneda y ver cuántas caras salen\n", "- La distribución es una binomial\n", "- La esperanza de una binomial es $np$, siendo $n$ el tamaño de la muestra y $p$ el valor a estimar\n", "- La desviación estándar de una binomial es $\\sigma = \\sqrt{np(1-p)}$\n", "\n", "Para una distribución binomial\n", "\n", "$error_s(h)=r/n$, por lo que $E[error_s(h)]=p$ \n", "\n", "por lo que el error en la muestra es un estimador no sesgado de p. \n", "\n", "- Para calcular la desviación estándar, sustituimos $p$ por nuestra estimación. \n", "\n", "\n", "$\\sigma_{error_s(h)} = \\sigma_{r/n} = \\sqrt{Var(r/n)} = \\sqrt{\\frac{1}{n^2} Var(r)} = \\frac{\\sigma_r}{n} = \\sqrt{\\frac{p(1-p)}{n}} \\approx \\sqrt{\\frac{error_s(h)(1-error_s(h))}{n}} $\n", "\n", "- Para calcular el intervalo de confianza, aproximamos la binomial con una distribución normal\n", "\n", "(Quien esté interesado en los detalles, puede consultar la sección 5.2 y 5.3 del libro de Mitchell)\n" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Precisión y Recuperación\n", "\n", "- El problema con el acierto y el error es que no tienen en cuenta el\n", "comportamiento en las distintas clases.\n", "- Si el 99% de las instancias son azules, la función constante azul tiene\n", "un acierto de 99%.\n", "- Se buscan alternativas para medir por clase: precisión, recuperación (_recall_)\n", "\n", "\n", "$$ precision = \\frac{V_p} {V_p + F_p} $$\n", "\n", "$$ recall = \\frac{V_p} {V_p+F_n} $$\n", "\n", " \n", "\n", "$V_p$ indica verdaderos positivos, es decir aquellos ejemplos que fueron clasificados correctamente\n", "\n", "$F_p$ indica falsos positivos, es decir aquellos ejemplos que fueron clasificados como positivos, pero eran negativos\n", "\n", "$V_n$ indica verdaderos negativos\n", "\n", "$F_n$ indica falsos negativos \n", "\n" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Precisión y Recuperación\n", "\n", "Ejemplo: tenemos 100 instancias ($n = 100$) donde evaluar una hipótesis $h$, y dos clases posibles {1,0} (consideramos a la clase 1 como la clase positiva).\n", "\n", "Podemos construir la siguiente tabla, llamada _matriz de confusión_:\n", "\n", "| A | h(x)=1 | h(x)=0 | Total |\n", "|-----|----------:|----------:|-------:|\n", "| y=1 | $48_{TP}$ | $12_{FN}$ | 60 |\n", "| y=0 | $5_{FP}$ | $35_{TN}$ | 40 |\n", "| | 53 | 47 | 100 |\n", "\n", "Precision (¿De los que h predijo positivos, cuántos lo eran?)\n", "\n", "$$\n", "\\frac{TP }{h(x)=1} = \\frac{48}{53} = 0.91\n", "$$\n" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Precisión y Recuperación\n", "\n", "Ejemplo: tenemos 100 instancias ($n = 100$) donde evaluar una hipótesis $h$, y dos clases posibles {1,0} (consideramos a la clase 1 como la clase positiva).\n", "\n", "Podemos construir la siguiente tabla, llamada _matriz de confusión_:\n", "\n", "| A | h(x)=1 | h(x)=0 | Total |\n", "|-----|----------:|----------:|-------:|\n", "| y=1 | $48_{TP}$ | $12_{FN}$ | 60 |\n", "| y=0 | $5_{FP}$ | $35_{TN}$ | 40 |\n", "| | 53 | 47 | 100 |\n", "\n", "Recall (¿Cuántos de los positivos pudo h predecir correctamente?)\n", "\n", "$$\n", "\\frac{TP }{y=1} = \\frac{48}{60} = 0.80\n", "$$\n" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Precisión y Recuperación\n", "\n", "¿Qué pasa si consideramos a 0 como la clase positiva? Es equivalente a repetir las medidas anteriores, pero con los números cambiados en filas y columnas.\n", "\n", "\n", "| A | h(x)=1 | h(x)=0 | Total |\n", "|-----|----------:|----------:|-------:|\n", "| y=1 | $35_{TP}$ | $5_{FN}$ | 40 |\n", "| y=0 | $12_{FP}$ | $48_{TN}$ | 60 |\n", "| | 47 | 53 | 100 |\n", "\n", "Accuracy (¿cuántos casos predice h correctamente?): $\\frac{TP + TN}{Total} = \\frac{35+48}{100} = 0.83$ \n", "\n", "Precision (¿De los que h predijo positivos, cuántos lo eran?): $\\frac{TP }{h(x)=1} = \\frac{35}{47} = 0.74$ \n", "\n", "Recall (¿Cuántos de los positivos pudo h predecir correctamente?): $\\frac{TP }{h(x)=1} = \\frac{35}{40} = 0.88$\n", "\n", "La precisión y el recall son siempre respecto a una clase positiva, el accuracy es una medida general. \n" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Medida-F\n", "- Combinando precisión y recuperación se obtiene la medida-F (donde $\\beta$ indica cuánta más importancia se le da al recall respecto a la precisión) : \n", "\n", "$$F_\\beta = \\frac{(1+ \\beta^2) \\cdot precision \\cdot recall}{\\beta^2 \\cdot precision + recall}$$\n", "\n", "- En el caso de $F_1$, la formula queda reducida a:\n", "\n", "$$F_1 = \\frac{2 \\cdot precision \\cdot recall}{ precision + recall}$$\n", "\n", "- La medida-F es la media armónica entre precisión y recall, e intenta combinar ambas en un sólo número. \n", "\n", "Para una interesante discusión sobre por qué se utiliza la media armónica y no la media aritmética, sugerimos este artículo: [The truth of the F-measure](https://www.cs.odu.edu/~mukka/cs795sum09dm/Lecturenotes/Day3/F-measure-YS-26Oct07.pdf) - Yutaka Sasaki" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Medida-F\n", "\n", "\n", "Ejercicio: complete la siguiente tabla, para una clasificación sobre 100.000 instancias\n", "\n", "\n", "| $V_p$ | $F_p$ | $F_n$ | $V_n$ | Prec | Recall | $F_1$ | Accuracy |\n", "| :---: |:---: | :---: | :---: | :---: | :---: | :---: | :---: |\n", "| 25 | 0 | 125 | 99850 | 1.00 | 0.17 | 0.29 | 0.999 |\n", "| 50 | 100 | 100 | 99750 |0.33 | 0.33 |0.33 |0.999 | \n", "| 75 | 150 | 75 | 99700 | 0.33 | 0.50|0.40|0.998 | \n", "| 100 | 50 | 50 | 99800 | 0.67 | 0.67|0.67|0.999 |\n", "| 150 | 100 | 0 | 99750 | 0.60 |1.00| 0.75| 0.999 |\n", "\n", "- ¿Qué sucede con la accuracy? ¿Y con los otros valores? ¿Cuál de los clasificadores eligiría?\n" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Matriz de confusión\n", "\n", "Las matrices de confusión son muy útiles para visualizar cómo se comporta una hipótesis en problemas con varias clases. En el siguiente ejemplo (tomado de [aquí](https://www.blackhc.net/blog/2019/mnist-by-zip/)), se muestra una matriz de confusión para un clasificador de dígitos escritos a mano (que no funciona muy bien, dicho sea de paso...). En este caso, los valores fueron normalizados, esto es, varían entre 0 y 1 (¿imagina para qué?)\n", "\n", "\"Drawing\"\n", "\n" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Titanic: Precision, Recall, medida-F\n" ] }, { "cell_type": "code", "execution_count": 43, "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Accuracy:0.787 \n", "\n", "Classification report\n", " precision recall f1-score support\n", "\n", " 0 0.77 0.94 0.84 202\n", " 1 0.85 0.54 0.66 127\n", "\n", " accuracy 0.79 329\n", " macro avg 0.81 0.74 0.75 329\n", "weighted avg 0.80 0.79 0.77 329\n", " \n", "\n", "Confusion matrix\n", "[[190 12]\n", " [ 58 69]] \n", "\n" ] } ], "source": [ "# Construimos un clasificador con el mejor parámetro, y entrenamos sobre todo el conjunto de entrenamiento\n", "measure_performance(X_test,y_test,clf_dt)" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Problemas multiclase\n", "\n", "- ¿Qué sucede cuando se tiene un problema multiclase (es decir, hay más de dos categorías)?\n", "- Las medidas anteriores siguen valiendo, si consideramos como \"positivos\" a las instancias que pertenecen a una clase, y \"negativos\" al resto (one-versus-all).\n", "\n" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Problemas multiclase\n", "\n", "Supongamos que tenemos tres clases a,b,c, y la siguiente matriz de confusión: \n", "\n", "| A | h(x)=a | h(x)=b | h(x)=c | Total |\n", "|-----|----------:|----------:|----:|-------:|\n", "| y=a | $20$ | $10$ | $10$ | 30 |\n", "| y=b | $3$ | $21$ |$6$ | 30 |\n", "| y=c | $0$ | $1$ | $30$ | 40 |\n", "| | 23 | 41 | 46 | 100 |\n", "\n", "Accuracy (general) : $\\frac{20+21+30}{100} = 0.71 $\n" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Problemas multiclase\n", "\n", "Supongamos que tenemos tres clases a,b,c, y la siguiente matriz de confusión: \n", "\n", "| A | h(x)=a | h(x)=b | h(x)=c | Total |\n", "|-----|----------:|----------:|----:|-------:|\n", "| y=a | $20_{TP}$ | $10_{FN}$ | $10_{FN}$ | 30 |\n", "| y=b | $3_{FP}$ | $21_{TN}$ |$6_{TN}$ | 30 |\n", "| y=c | $0_{FP}$ | $1_{TN}$ | $30_{TN}$ | 40 |\n", "| | 23 | 41 | 46 | 100 |\n", "\n", "$Precision_A : \\frac{TP}{h(x)=a} = \\frac{20}{23} = 0.87 $\n", "\n", "\n", "$Recall_A$ : $\\frac{TP}{y=a} = \\frac{20}{30} = 0.67 $\n", "\n", "\n", "(Ejercicio, verifique $Pr_b=0.51$, $R_b=0.7$, $Pr_c=0.65$, $R_c=0.75$)\n", "\n" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Problemas multiclase\n", "\n", "- Esto nos da una medida por cada clase. Existen diferentes formas de resumir esa información:\n", "\n", " Se calcula la medida por clase, y luego se promedia los valores obtenidos (macro average)\n", " \n", " Se calcula la medida por clase teniendo en cuenta el aporte de instancias cada clase (micro average)\n", " \n", "En el ejemplo anterior: \n", "- La macro-average de la precision será $ \\frac{(Pr_a + Pr_b + Pr_c)}{3} = \\frac{0.87 + 0.51 + 0.65}{3} = 0.67 $\n", "- La micro-average será $ \\frac{20+21+30}{23+41+46} = 0.71 $ (en un problema multiclase, la micro es igual a la accuracy!)\n", "\n", "\n", "\n", "- Qué medida es más \"útil\", como toda medida, **depende de lo que queremos evaluar**:\n", " - La micro-average da más peso a las clases grandes en el análisis general\n", " - La macro-average permite evaluar mejor que tan \"equilibrado\" es el comportamiento de mi clasificador \n", " - En lo posible, reportar ambas y analizar según mi problema\n", " \n", "Lectura recomendada: [Should I make decisions based on micro-averaged or macro-averaged evaluation measures?](https://stats.stackexchange.com/questions/156923/should-i-make-decisions-based-on-micro-averaged-or-macro-averaged-evaluation-mea)\n" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Problemas multietiqueta\n", "\n", "\n", "- En un problema multietiqueta, hay más de una clase asociada a cada etiqueta, como en el siguiente ejemplo\n", "\n", "| Instancia | Clase | Predicción |\n", "| --- |:---: | :---: | \n", "| 1 | A,B | B,C | \n", "| 2 | A,B,C | A,C,D |\n", "| 3 | A,B | A,B |\n", "\n", "- Se pueden calcular las medidas utilizando solamente los ejemplos de cada clase, sin importar el resto. Esto permite utilizar las medidas mencionadas anteriormente\n", "\n", "- Se pueden aplicar otras medidas, como promediar el índice de Jaccard $IJ(A,B) = |A \\cap B| / |A \\cup B|$\n", "\n", "- En el ejemplo, $IJ(1)=1/3$, $IJ(2)=1/2$, $IJ(3)=1$, y por lo tanto el índice de Jaccard promedio será $0.61$\n" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "skip" } }, "source": [ "### Variables ordinales\n", "\n", "¿Qué pasa si queremos predecir variables ordinales (e.g. cantidad de córners de un partido)? \n", "\n", "\"Drawing\"\n", "\n", "Posible estadística: accuracy_n (predecir el valor d, o hasta d+n)\n" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Línea base y línea máxima\n", "\n", "- ¿Cómo sabemos si el resultado que obtuvimos es \"bueno\", o \"razonable\"?\n", "\n", "- El resultado depende del problema. \n", "\n", "- Siempre es bueno tener una línea base: una solución anterior sencilla, o un clasificador que elige siempre la clase más probable o según la distribución del conjunto de entrenamiento\n", "\n", "- También es útil (si es posible) tener una línea de tope, sobre todo en problemas donde no hay antecedentes. Típicamente, se pide a humanos que actúen como clasificadores y se evalúa su performance. \n" ] } ], "metadata": { "celltoolbar": "Slideshow", "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.13" } }, "nbformat": 4, "nbformat_minor": 4 }